Re: Lua plans to add meta-programming support?

[Xavier Wang <weasley.wx@...>]

I also think of to use lpeg to do this. but I found lpeg has it's limit:
 - as you said, we need some trick to preserving the line information, etc.
 - and a killer issue is, lpeg parse the *whole* code, not the code
chunk, we can't offer a lua_Reader to lpeg!
 - using lpeg to reimplement a lua lexer may introduce bugs, and can
hardly produce the same error message as Lua.

that's why I think of a general purpose token filter to lua itself, it
doesn't have such issue. we can make metalua/luaMacro/lc top of it.
using it, we can split the compile time and run time in Lua, that open
the door of lua meta-programming. we can easily implement the macro in
Lua, what Roberto was thought.

maybe a discuss about what a token filter will do and how to provider
this service is necessary. e.g. maybe a token filter will produce the
ast of Lua code, and serve it to metalua.

2012/3/10 steve donovan <steve.j.donovan@gmail.com>:
> On Sat, Mar 10, 2012 at 8:08 AM, Xavier Wang <weasley.wx@gmail.com> wrote:
>> Now we are discussing some semantics proposals in Lua. I believe that
>> the main solution is makes Lua easy to meta-programming. i.e., add
>> some hooks to llex, makes Lua can callbacks on lexing).
>
> We did have the token-filter patch by lhf, which did pretty much what
> you suggested. I based the first version of LuaMacro on top of it, but
> requiring a patched Lua is irritating, so the current version uses
> LPeg to do the tokenizing [1]  (The tricky part is preserving original
> line information so that error messages and error tracebacks make
> sense)
>
> These are lexically-scoped macros, so they don't contaminate the whole
> global namespace.  For instance, consider replacing mytable[#+1] with
> mytable[#mytable+1] which was suggested recently.  That is, you can
> write code like this:
>
>  require_ 'rawhash'
>
>  Tab mytable, another
>
>  t = {1,3}
>
>  -- Here # is short for #mytable
>  mytable[#+1] = 1
>  mytable[#+1] = 2
>
>  -- without indexing, behaves just like a table reference
>  assert(type(mytable)=='table')
>
>  -- it is still possible to use #t explicitly
>  assert(mytable [#]==mytable[#t])
>
>  assert(mytable[#-1] == mytable[1])
>
> Here 'Tab' is a macro which generates locally-scoped macros, and the
> implementation is here [2]
>
> Also works interactively, where you have access to other things like
> lambda short-forms:
>
> C:\Users\steve\lua\LuaMacro\tests> luam -lrawhash -i
> Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> Lua Macro 2.3.0 Copyright (C) 2007-2011 Steve Donovan
>> Tab mytable
>> mytable = {10,20,30}
>> = mytable[#]
> 30
>> f = \x(2*x)
>> = f(2)
> 4
>
> There are limitations - this is smart substitution but it is still on
> the lexical level. I cannot implement the most commonly suggested |x|
> 2*x syntax for short lambdas because it's hard to know where the
> expression ends without parsing it.  Some of the freedom of
> token-filtering is also lost, so I can't show you a filter that
> replaces newlines in tables with commas (I'm thinking of an elegant
> yet efficient way of doing this)
>
> steve d.
>
> [1] https://github.com/stevedonovan/LuaMacro
> [2] https://github.com/stevedonovan/LuaMacro/blob/master/tests/rawhash.lua
>