Re: [ANN] LuaMacro 2 - a lexical macro preprocessor for Lua

[steve donovan <steve.j.donovan@...>]

On Mon, May 2, 2011 at 3:40 PM, Fabien <fleutot+lua@gmail.com> wrote:
> It has the interest of being conceptually simple, and it should be easy to
> hack simple syntax tweaks based on this. Hacking on strings is simple,
> because strings are a simple and well-known data structure. The price you
> pay is, it's really hard to get anything that really works:

That is my feeling - I don't think that string transformations are
precise and robust enough. What I did like was the idea of a custom
loader which would preprocess-and-load files of a given extension.

> Programs can be seen at different structuring level: strings, token streams,
> parse trees, abstract trees. The less structured the approach, the easier it
> is to do a quick and dirty job; the more structured approaches allow to
> write something correct, but generally have a steeper learning curve.

For me, the sweet spot here is 'token streams' but I'm very aware of
its limitations. One cannot do arbitrary syntax transformations (for
instance, one can't do the short lambda form '|x| expression-in-x'
because there is no way of knowing when an expression ends without
parsing.)  But the 'surface' transformations one can do are useful and
relatively easy,

>  I would think that new
> syntaxes are mainly intended to ease maintenance and readability, but I fear
> that the brittleness inherent to low level approaches more than outweigh any
> gain.

That would be the question.  You have always emphasized that good
macros are much harder to write than bad macros, but the payoff is
justified.  For instance, a good 'forall' lexical macro which converts
'forall n in {10,20,30} do' into its generic-for equivalent has got to
check that there is indeed a loop variable, followed by 'in' and an
expression which is everything up to 'do'.

> How much confidence would you put in code that's rewritten by half a
> dozen preprocessor macros, which don't even have a way to express the notion
> of scope?

At least my current set of lexical macros can be scoped. ;)  The moral
of the story is to use as few syntax modifications as possible to
achieve one's goals.  It is probably like the old Unix koan 'When
should you optimize?' - the immediate answer is "Don't" and the
advanced answer is "Not yet".

steve d.