The Lua lexer - what am I missing?

["Matt Holmes" <kerion@...>]

Greetings,

I am attempting to read and understand the Lua lexer/parser, and I have run
in to a slight snag. For the most part I understand the concept of parsing
for tokens (for instance >= is TK_GE), but what I am missing is the use of
the underscore ( _ ) character as a delimiter to begin searching for a
reserved word or identifier name. For instance, in luaX_lex:

case '_': goto tname;

        tname: {  /* identifier or reserved word */
          TString *ts = luaS_new(LS->L, readname(LS));
          if (ts->marked >= RESERVEDMARK)  /* reserved word? */
            return ts->marked-RESERVEDMARK+FIRST_RESERVED;
          seminfo->ts = ts;
          return TK_NAME;
        }

why would LS->current ever be an underscore? I know I for one do not write
an underscore before every keyword or identifier name. Have I utterly missed
something here? Is this some standard for lexical parsers that I may have
passed over in my research?

I looked to see if some kind of replacement was done on white space,
replacing all spaces with underscores, but I saw no such operation, nor
would I understand the point if I did. I did not see that the zgetc() macro
did any type of conditional return for whitespace either.

Obviously I am missing something fundamental here. Could someone clue me in
please? :)

Matt "Kerion" Holmes