Re: cgit-lua: to jit or not to jit

[William Ahern <william@...>]

On Tue, Jan 14, 2014 at 02:25:20AM +0100, Jason A. Donenfeld wrote:
> Hi folks,
> 
> Over at the cgit project [1], we're in the process of adding Lua
> support. Things are going swimmingly, and I've just merged to the
> master tree some commits adding full lua support [2] to our filter API
> [3] and an example script [4]. At the moment we've got some moderately
> messy logic in our makefile [6] that autodetects if LuaJIT is
> installed, and if it isn't, falls back to mainline Lua.
> 
> What I'm wondering is -- what is the purpose of keeping around support
> for mainline Lua? Why shouldn't we just go with LuaJIT and be done
> with it? I'm unable to find any advantages mainline Lua has over
> LuaJIT, which is why I'm emailing these lists. What features am I
> forgetting about that would make some people prefer mainline Lua over
> LuaJIT?

The 2GB memory limit on 64-bit was a huge blocker at my work. We almost
immeditely ran afoul of it and have chosen to stabilize on Lua 5.2 for one
of our cloud services. (On 16-way SMP, 32GB RAM boxes it's more important to
be able to scale up and out then to have JITd loops, especially when the
most performance sensitive code is in C anyhow.)

> An additional motivation for wanting to go to LuaJIT is the FFI
> library that comes with it [5]. It seems like this would make
> packaging and swapping scripts a lot easier. For example, in the
> gravatar script [4], I depend on luacrypto, which some users might not
> want to install or have easily available. With FFI, I could instead
> just wrap OpenSSL (which cgit already depends on). But perhaps there
> are some obvious downsides to this approach that I also am missing.

I suspect that if you ended up trying to write a full OpenSSL wrapper using
LuaJIT's FFI that you wouldn't save very much time and effort. I say that as
someone having written the most comprehensive OpenSSL bindings in Lua:

	http://25thandclement.com/~william/projects/luaossl.html

Things might be different if you just bound one or two interfaces. LuaJITs
FFI excels at that sort of ad hoc interface binding. But for full-blown
bindings things get more complicated. OpenSSL, for example has a ton of
preprocessor generated tags. Also, string and buffer management can be made
significantly more efficient in C, so when you're aggregating results into a
large buffer before pushing onto the Lua stack, it's easier to get good
performance with C code. For larger or complex modules the payoffs with
LuaJIT FFI diminish. So it all depends on your context. How sophisticated
will your bindings be?

Because Lua is emebedded in your project it's not that big of deal to just
settle on LuaJIT. Having to deal with the different versions is more of a
problem for a module writer, where you run into problems with differences
across versions and implementations--e.g. GC (Lua 5.2 has ephemeron tables)
or internals (e.g. FILE* handles).