Re: Luma errors

[Fabio Mascarenhas <mascarenhas@...>]

On Mon, Apr 11, 2011 at 1:33 PM, Roberto Ierusalimschy
<roberto@inf.puc-rio.br> wrote:
>> I wrote Luma against a pretty old version of LPEG, 0.7. I have updated
>> luma.re to work with LPEG 0.10, forking LPEG's re.lua. The changes are
>> simple:
>>
>> * allow _ as a non-terminal name
>> * search for non-terminals in the definitions first (instead of
>> requiring definitions to be prepended by %)
>>
>> These changes make writing and composing grammars much more convenient.
>
> Would you mind explaining how these changes make writing and composing
> grammars *much* more convenient?

The second change lets the grammar writer refactor a non-terminal to a
definition without adding % to all of its uses. The first stab at a
grammar can be written in RE, then parts of the grammar that are
useful to other grammars can be made into definitions. which can then
be reused in other grammars with the same syntax as other
non-terminals.

The first change, when combined with the second, just lets Luma offer
_ as a predef for Lua whitespace (in the lexer sense) for the user's
grammars, which is useful in a lot of grammars and less obstrusive
than %space or %s.

You might remember that RE just used names for both non-terminals and
predefs, and also allowed _, but it had a bug :-)
(http://lua-users.org/lists/lua-l/2008-01/msg00492.html). The fork
originally just fixed this bug, but when 0.8 split non-terminals and
defs I decided to just keep using the patched RE from LPEG 0.7. When
LPEG 0.9 removed accumulator captures I just had forgotten about
Luma's RE until Henk tried to use it.

Cosmo was also using a patched LPEG 0.7 to fix this bug and add a
"state capture", when LPEG 0.8 introduced argument captures I rewrote
Cosmo's RE grammar to use the new RE syntax so I could just use stock
LPEG, but Cosmo's users are not supposed to be messing with its
grammar.

> (I wonder whether it is worth forking re.lua only because of what looks
> like cosmetic details.)

It is cosmetic, but it is also a question of what it is designed for.
I understood the rationale for the % prefix when it was introduced
(make definitions look like the predefs of Lua patterns), and the way
RE works is great for a replacement for Lua patterns, which I think is
its itended usage. But the purpose of the RE embedded in Luma is to
make it easier to write the syntax of its macros, as each macro can
have its own mini-language. I mostly stopped working on it after the
"summer of macros" in the Lua list, but if I had kept working on it I
would have added support for making it easier to writing grammars that
handle errors in its fork of RE.

> -- Roberto
>

--
Fabio