Documentation, Literate Programming and the Real World

[sur-behoffski <sur_behoffski@...>]

(As usual, sorry for non-threading... the digest helps me cope with things.)

I've worked on a major system where the primary author was a great fan of
Knuth's Literate Programming ideas.  Seeing Knuth's "Web" (I think) tool
which completely inverted the coding-versus-documentation relationship, he
came up with his own variant, called "FunnelWeb".  This dates to around the
mid-80s to mid-90s (I think).

In this system you wrote a series of essays explaining what you were doing,
introduced tagged code snippets as part of the essay exposition, and then,
still using exposition, used directives naming the tags to pull the pieces
together, until you had a complete program.  Editing was always done on the
essay version of the code, and a preprocessor automatically took care
of extracting the snippets and preparing the code in a format suitable for
the compiler.

The product was very successful, but the company eventually decided to
switch around from FunnelWeb to the more conventional
documentation-in-source tools seen today (possibly because the more
mainstream tools had matured and could produce HTML etc., whereas this
would require a separate project, and ongoing maintenance, for what was
essentially an in-house tool).

This exposure to Literate programming has influenced my programming
style -- I did some serious work in Modula-2 at one point, and, while
there were parts of the language that I strongly disliked, there were other
portions where the discipline it imposed (especially the
interface/implementation divide) impressed me.  So, I tend to write all my
code in other languages with a soft Modula-2 discipline overlaid to help
me manage.

On the code comment front, I rejected Web/FunnelWeb as too alien for most
people, but tried to come up with a soft literate-inline version.  Also, I
decided to used Camel Case names, mainly as I believed that they are
generally easier to read than all-uppercase or all-lowercase names.

I strove to limit code nesting, ruthlessly using 8-column indents so that
highly-nested code was strongly discouraged (as it ran off the right-hand
side of the page), to break code into paragraphs, and to demand that each
paragraph have a preceding comment.  In this context, Lua's lack of a
"continue" statement (until goto came along) was quite frustrating.

For each paragraph, I write a comment as if I'm explaining the code to an
abstracted group of people as I'm going, using "we" quite often.

For example, some stripped-down regular expression lexing, in C):


------------------------ (cut here) --------------------------

bool
ProcessREPattern(char *pPattern, int PatternLen)
{
        /* [...] Some initialisation code elided for brevity [...].  */

        /* Loop through the pattern bytes, converting to tokens, including
           handling special characters as appropriate.  */
        for (;;) {
                /* Have we reached the end of the pattern?  */
                if (pPat == pPatEnd) {
                        /* Yes, break out of this loop so we can do some
                           finalisation and sanity checking.  */
                        break;
                }

                /* Get the next character of the pattern.  */
                Ch = *pPat++;

                /* Is this the start of a bracket expression?  */
                /* FIXME:  Escape may toggle character interpretation here.  */
                if (Ch == '[') {
                        /* Yes, this is a big job:  Use BracketExp().  We also
                           need to hand in current position, and to get back
                           how many chars the bracket expression consumed.  The
                           function outputs tokens directly, as required.  */
                        Position_t Position;

                        /* Wrap the BracketExp call with context handover/handback
                           code.  */
                        Position.pPatIn = pPat;
                        Position.pPatEnd = pPatEnd;
                        Position.pTokensIn = pTokens;
                        if (! BracketExp(&Position)) {
                                /* Fatal error found during processing.  */
                                return false;
                        }
                        pPat = Position.pPatOut;
                        pTokens = Position.pTokensOut;

                        /* Finished dealing with this component.  */
                        continue;
                }

                /* Extremely simplified code:  Merely emit character as a token.  */
                *pTokens++ = TOKENISE(Ch);
        }

        /* [...] Cleanup/checking code elided here. [...]  */

        /* Finished processing pattern successfully.  */
        return true;
} /* ProcessREPattern */