Lua versus Ravi versus Luajit - another perf test

[Dibyendu Majumdar <mobile@...>]

Hi,

After adding some more bytecodes to the JIT compiler I was able to run
a test that is more complex than the previous tests. The results are
below:

C:\github\ravi\ravi-tests>\luajit\luajit.exe mandel.lua 4000
1.633

C:\github\ravi\ravi-tests>\lua-5.3.0\src\build\Release\lua.exe mandel.lua 4000
21.247

C:\github\ravi\ravi-tests>..\build\Debug\lua.exe mandel1.ravi
5.79

Due to limitations in supported bytecodes I had to hard-code the 4000
parameter in mandel1.ravi.


The test programs can be found at:
https://github.com/dibyendumajumdar/ravi/tree/master/ravi-tests

My conclusion from the work done so is that the JIT compilation of Lua
using LLVM is a worthwhile initiative. I feel that it will not be
possible to beat Luajit for many use cases because of certain
differences in approach and implementation.

1) Luajit uses an optimized representation of values. In Lua 5.3 and
in Ravi, the value is 16 bytes - and many operations require two loads
/ two stores. Luajit therefore will always have an advantage here.

2) More work is needed to optimize fornum loops in Ravi. I have some
ideas on what can be improved but have parked this for now as I want
to get more coverage of Lua bytecodes first.

3) Luajit obviously is significant smaller in size when LLVM is added
to the comparison. Using LLVM will tend to preclude Ravi's JIT engine
from being useful in constrained devices - although ahead of time
compilation could be used in such cases.

One of my goals is to keep the code base relatively simple, and avoid
very exotic optimizations. Code maintainability is very important to
me. I also hope that keeping the implementation simple, and having it
well documented will encourage others to participate and help in the
effort.

Compatibility with Lua 5.3 is also important to me. However, I don't
know what changes 5.4 will bring so it is difficult to foresee whether
I can keep up with changes in mainstream Lua.

LLVM appears to be best choice for JIT compilation despite its
drawbacks. Firstly it has momentum and very active / well sponsored
implementation team. I looked at various other alternatives but none
has the same user base / momentum. LLVM also appears to be
continuously improving - so all the advancements made in LLVM will
accrue for free. Only cost is that they keep changing APIs with every
major release (somewhat like Lua!) so it is pain to upgrade.

Thanks and Regards
Dibyendu