[ANN] LuaVela, an implementation of Lua 5.1 based on LuaJIT 2.0

[Anton Soldatov <igelhaus@...>]

Dear all,

tl;dr After several years of in-house development and production
usage, IPONWEB releases LuaVela, an implementation of Lua 5.1 based on
LuaJIT 2.0. For now the project supports Linux x86-64 only. The
license is MIT.

Please check out https://github.com/iponweb/luavela, patches and bug
reports are welcome.

A longer version:

I'm glad to announce the first public release of LuaVela, an
implementation of Lua 5.1 based on LuaJIT 2.0. The project was started
as a fork of LuaJIT 2.0 in 2015. The primary goal was to fix the
notorious 2Gb memory limit on the x86-64 platform. Eventually the
project accumulated several features that we would like to share with
the community.

List of major features:

* Full support for 64-bit memory without any tricks or hacks in the
interpreter and JIT compiler;

* "Sealing": An ability to hide some data from the garbage collector.
In IPONWEB, we use this generation-like (or, better, Eden-like) trick
to mark data with the same lifetime as the application instance itself
reducing overall pressure on GC;

* Immutability: Data structures may be (recursively) marked immutable
in run-time. This implemented via an extension API, the syntax of the
language is unaffected;

* Coroutine timeouts: There are C-level extension APIs that allow to
control the life time of coroutines – once a coroutine runs for too
long, it is terminated by the virtual machine;

* Some new optimizations in the JIT compiler (but some of them are not
brand new if one compares with LuaJIT 2.1);

* New C- and Lua-level extension APIs;

* Platform-level sampling profiler;

* Memory usage profiler;

* Platform-level code coverage.

Apart from that:

* CMake is used as a build system for the project;

* 6 test suites are bundled with the project: Lua 5.1 test suite,
LuaJIT test suite, CERN MAD test suite (partially), lua-Harness test
suite and two suites written inside IPONWEB (for testing at Lua- and
C-level, respectively);

* Documentation bundle is included into the release, too. All the docs
are in the RST format and `make docs` will build you the HTML version
if you have Sphinx installed. This one is the last thing we prepared
before the release, so apologies for some mishmash there, we will sort
it out eventually.

Feel free to ask any questions, but here is a FAQ list based on what
I've been asked during my talks, etc.:

Q: What does "Vela" in LuaVela mean?

A: It is Vela constellation.

Q: What does "uJIT" mean in the docs and the core code base?

A: This is an internal name of the project in IPONWEB. Unfortunately,
we could not simply strip it off prior to the public release, so
apologies for possible confusion. LuaVela is the preferred public name
of the project, uJIT is something like an internal "code name".

Q: What is the level of compatibility with Lua 5.1?

A: The same as for LuaJIT 2.0. But we wish we could make the
implementation closer to the PUC-Rio implementation.

Q: Why not LuaJIT 2.1 + LJ_GC64?

A: As far as I know, this solution got stabilized by 2016-2017. Alas,
we needed something working earlier. Besides, some experiments we ran
in 2015 with LuaJIT 2.1 showed performance degradation for our cases.

Q: What about support for Lua 5.2+?

A: Unfortunately, we do not have resources to implement it on our own.
However, patches are welcome. We definitely do *not* plan to increase
the compatibility gap with the PUC-Rio implementation. E.g. dropping
support for C API or any initiative like this is not an option for the
project.

Q: What about support for other OSes and other platforms than Linux x86-64?

A: I doubt that 32-bit platforms will be supported. Regarding other
operating systems, we do not have resources to implement it on our
own. However, patches are welcome.

And last, but definitely not least. Many people have worked hard to
make this public release happen. I would like to acknowledge some of
them personally:

* Igor Ehrlich, the godfather of the project who started it and did an
unbelievable amount of work during the project's early development and
establishment phases. In 2016, he gave an excellent talk (sorry, in
Russian only) about our reasons to start an own implementation [1];
* Maxim Bolshov, who put much of his expertise into JIT-related things;
* Ilya Dailidyonok, who, apart from delivering various features, did a
fantastic work polishing the code base at all levels preparing it for
the public release.

P.S. For those who wish more context, here is a couple of my talks on
the matter:

* "Challenges Building Yet Another Lua Implementation", at Lua in
Moscow 2017 (in English) [2];
* "Rewriting LuaJIT: Why and How?", at Lua Workshop 2018. The talk is
in English, but I as far as I know there are slides only [3].

[1] https://www.youtube.com/watch?v=64c4ihDl6MM
[2] http://lua.moscow/conf/2017-03-LuaInMoscow/index.html#soldatov
[3] https://www.lua.org/wshop18/Soldatov.pdf

-- 
Med vennlig hilsen,
Anton Soldatov