Re: [ANN] LuaJIT-2.0.0-beta1

[Vyacheslav Egorov <mister.aleph@...>]

> You might implicitly achieve this by formatting
> your generated Lua code in such a way that line numbers in the
> original and generated sources match up, but this might not be always
> possible.

Ah, I am always forgetting about debug info...

If I remember correctly (I was the one who initially contributed
simple version of debug info handling to metalua, but it was long ago)
when you create piece of AST by quasiquotes in A.mlua and then splice
it into AST from B.mlua source code positions pointing to A.mlua will
be preserved in resulting AST. So indeed there is no way to format
resulting AST to preserve debug info.

--
e.v.e



On Tue, Nov 3, 2009 at 12:20 PM, David Manura <dm.lua@math2.org> wrote:
> On Mon, Nov 2, 2009 at 11:22 PM, Vyacheslav Egorov wrote:
>> it seems that there is no bytecode frontend in Lua JIT now, so it is
>> not possible to create Lua JIT backend for metalua.
>>
>> The only option is to provide a source code backend for metalua. But
>> as far as I remember metalua can produce irreducable control flow
>> graphs (AST contains `Goto constructs). So translation into source is
>> not a straightforward task.
>
> True.  Conceptually, Metalua does these two steps:
>
>  (A) translates a source code string--either plain Lua or Lua with
> extended syntax--into an AST of the form given in [1].  This step is
> implemented in pure Lua code, and the emitted AST is in the form of
> Lua tables.
>
>  (B) translates the AST into Lua bytecodes.  This step is also
> implemented in pure Lua code based on Yueliang (a port of the the Lua
> compiler to Lua).  However, the output is normally standard Lua 5.1
> bytecodes.
>
> So, you should be able to use Metalua under LuaJIT2 to perform steps A
> and/or B provided you don't use any Metalua extensions modules
> implemented in Metalua without first rewriting them in terms of
> standard Lua.  However, I understand you can't run those bytecodes in
> LuaJIT2 without perhaps some ugly portability layer.
>
> You may, however, replace step B with your own generator.  This could,
> for example, output LuaJIT2 bytecodes if LuaJIT2 added such an
> interface for reading those bytecodes.  Alternately you could emit
> standard Lua source.  Metalua 0.5 already has a module to do that
> latter [2] for standard Lua ASTs.  This module is implemented in
> Metalua, but it is easy enough to rip out the Metalua syntax
> extensions and replace them with standard Lua.
>
> Now, emitting standard Lua source has a couple issues, each of which
> can be addressed in a few different ways.
>
> The first problem is that the Metalua AST [1] has a couple of
> extensions outside of standard Lua syntax.  These are `Goto/`Label and
> `Stat nodes.  `Goto/`Label are for gotos.  `Stat executes statements
> inside of expressions, which can frequently occur upon expanding
> syntax extensions and macros.  Though you can achieve `Stat with
> closures, the purpose of `Stat is to avoid closures for efficiency
> sake.  `Stat nodes perhaps can be avoided.  I had once written a
> source code optimizer that can inline function calls [3], and the
> primary motivation of that was to eliminate the need for special
> bytecode generation for `Stat nodes in Metalua.  Similarly, maybe
> LuaJIT2 would negate these efficiency concerns as well.  The problems
> handling `Goto/`Label still remain, but I suppose there are ways to
> automatically translate gotos in plan Lua code or even patch Lua to
> support gotos.  The Clue discussions [5-6] went into that.  This
> problem is mitigated, however, in that that not all Metalua extensions
> emit `Goto/`Label nodes.  If you avoid those extensions or rewrite
> them in such a way not to emit these nodes, you won't have a problem.
> The same can be said about `Stat.
>
> The second problem is that standard Lua source does not allow emitting
> meaningful debug information like line numbers are symbol names.  This
> problem has been mentioned a number of times in the context of Lua
> source preprocessing.  You might implicitly achieve this by formatting
> your generated Lua code in such a way that line numbers in the
> original and generated sources match up, but this might not be always
> possible.  There have been suggestions to allow pragmas to be inserted
> into the Lua source to alter line numbers somewhat like the C
> preprocessor [6], and something similar might be done for symbols.
> This is a second reason why Metalua implements its own bytecode
> generator.  This second problem is likely easily solved by a patch to
> the Lua parser though.
>
> Though I agree it is helpful to think of Lua source as its own type of
> portable bytecode, a few Lua syntax extensions as described above
> could make it more effective in this regard.
>
> [1] http://metalua.luaforge.net/manual006.html#toc17
> [2] http://github.com/fab13n/metalua/blob/0.5-branch/src/lib/metalua/ast_to_string.mlua
> [3] http://lua-users.org/wiki/SourceOptimizer
> [5] http://lua-users.org/lists/lua-l/2008-07/msg00114.html
> [6] http://lua-users.org/lists/lua-l/2008-07/msg00238.html
> [7] http://lua-users.org/lists/lua-l/2009-05/msg00288.html
>