Re: A library to build Lua AST

[Elijah Frederickson <elijah.frederickson@...>]

A Lua parser (for 5.2) which generates a full AST, plus has options for refactorization is LuaMinify[1]. It is written in pure lua, no C/LPEG.

It can easily be changed to parse Lua 5.1 by removing the 5.2 keywords from the lexer/parser.

I highly recommend this project (perhaps because I helped work on it ;).

[1] - https://github.com/stravant/LuaMinify/

On Sat, Sep 7, 2013 at 9:43 AM, Richard Hundt <richardhundt@gmail.com> wrote:

On 07.09.2013, at 13:00, Alek Paunov <alex@declera.com> wrote:

> On 06.09.2013 21:22, Alexander Gladysh wrote:
>> I need to build Lua 5.1 AST for a bunch of files — as a bunch of
>> nested plain Lua tables.
>>
>
> ...
>
>>
>> Is there something out there that will suit my desires? Or what I can
>> adapt reasonably well?
>>
>
> Just to be sure that you have noticed this too (mentioned recently in the LuaJIT list):
>
> https://github.com/richardhundt/nyanga/commit/2804a70c
>
> Kind Regards,
> Alek
>

Nyanga doesn't parse vanilla Lua though (I can't tell if that's part of your requirements or not). If you ignore some fat (libuv dependency and runtime) you'd end up with something that lets you build a Lua AST programmatically.

You could use just the builder which you could use like so:

local B = require('builder')
local tree = B.chunk{
  B.expressionStatement(
     B.callExpression(
        B.identifier("print"), { B.literal("Hello World!") }
     )
  )
}

The builder validates the structure against a schema of the Lua syntax as you build it, so that makes it pretty hard to generate invalid code.

You then have a couple of options. You can either feed the tree into two kinds of code generators, a Lua source generator, and a LuaJIT2 bytecode generator, or you can dump the tree as JSON (which is probably closer to what you need):

local util = require('util')
print(util.dump(tree))

The above produces:
{
    "kind": "Chunk",
    "body": [
        {
            "kind": "ExpressionStatement",
            "_expression_": {
                "kind": "CallExpression",
                "arguments": [
                    {
                        "kind": "Literal",
                        "value": "Hello World!"
                    }
                ],
                "callee": {
                    "name": "print",
                    "kind": "Identifier"
                }
            }
        }
    ]
}

The AST is heavily inspired by the mozilla parser API [1].

You could of course tweak the dump routine to serialize this into valid Lua tables.

Cheers,
Richard

[1] https://developer.mozilla.org/en-US/docs/SpiderMonkey/Parser_API

--

~ mlnlover11 ~

https://github.com/mlnlover11