Re: [ANN] Development announcement - Ravi/Lua Compiler project

[Dibyendu Majumdar <mobile@...>]

On Sun, 8 Nov 2020 at 21:59, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
>
> On Fri, 21 Feb 2020 at 21:55, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
> >
> > Lua's inbuilt parser and code generator is a work of art, very compact
> > and low overhead but extremely fast. It uses minimal memory and
> > produces bytecodes as it parses the source code (single pass
> > compiler). This is great for Lua and Ravi given the use cases of these
> > languages, but makes the parser and code generator quite hard to
> > understand, play with, or reuse in tools such as IDEs. It also makes
> > it harder to perform any advanced type checking or performance
> > optimizations.
> >
> > I am pleased to announce a new project to create a Lua/Ravi parser and
> > code generator that is not a replacement for the default one in
> > Lua/Ravi but can be used for more specialised code generation, as well
> > as as a means of understanding how the parser and code generator
> > works. My hope is that it will enable the study of Lua implementation
> > in educational institutions, as I intend to use classical methods of
> > compilation. Moreover, ease of maintenance is a bigger priority than
> > sheer memory or performance efficiency.
> >
> > The project is currently under development. You can view the code at:
> >
> > https://github.com/dibyendumajumdar/ravi-compiler
> >
>
> I have been working on and off on this new compiler project. I am
> pleased to say that it has now got to a stage where I can use it
> inside Ravi for a subset of code.
> Although a lot more work needs to be done, this effort is already
> producing better results than the existing JIT compiler in Ravi.
>
> Key points:
> * The pipeline starts from Lua/Ravi source instead of bytecodes
> * The compiler creates AST during the parse phase
> * This is converted to Linear IR
> * From the Linear IR C code is generated
> * Finally this C code is compiled using MIR JIT backend
>
> You can see some examples of what works and the output the compiler
> generates here:
>
> https://github.com/dibyendumajumdar/ravi/tree/master/tests/comptests


One advantage of compiling from Lua/Ravi source is that it is now
possible to generate code ahead-of-time (AOT).
I was able to create a shared library from a test script and execute
this from Ravi.
Fortunately the same generated code can be run with the JIT compiler
or AOT compiler.

I have updated the outputs of the tests above so interested folks can
see what is generated.

Apart from completing full support of Lua language features, there is
still work to be done to improve the compiler. Ideally there should be
a type inference engine so that user do not need to add static
annotations except where necessary.

Regards