lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

For any purpose other than destructive indentation, it would be nice to have both the whitespace and comments ("greyspace") associated with the token stream. It should be possible to write a lua-lex-cat program equivalent to cat for (at least) well-formed Lua programs.


On Jun 10, 2017 2:35 PM, "Luiz Henrique de Figueiredo" <> wrote:
> An option to NOT discard the comments (or perhaps a secondary api call?) ???sounds like a great little 'tweak' to the core library that would provide a desirable benefit. How much effort would that so called 'little tweak' be (just curious, not volunteering)?

Not much effort at all, it seems. Here are the diffs in llex.c from Lua 5.3.4:

<             read_long_string(ls, NULL, sep);  /* skip long comment */
>             read_long_string(ls, seminfo, sep);  /* skip long comment */
>             return 'C';
<           next(ls);  /* skip until end of line (or end of file) */
>           save_and_next(ls); /* skip until end of line (or end of file) */
>         {
>           TString *ts;
>           ts = luaX_newstring(ls, luaZ_buffer(ls->buff),
>                                   luaZ_bufflen(ls->buff));
>           seminfo->ts = ts;
>         }
>             return 'c';

The patched code returns 'C' for a long comment and 'c' for a short
comment, with the contents of the comment in seminfo->ts. There is a
small bug that the first char in long comments is dropped but it is
probably easy to fix; I haven't tried.

I just added support for this to proxy.c in ltokenp and it works:

  switch (t)
    case 'C':
    case 'c':
    case TK_STRING:
    case TK_NAME: