Re: Question about LuaJIT's future FFI/Struct Extension

[Mike Pall <mikelu-1003@...>]

Peter Cawley wrote:
> Along the same lines, I noticed that in a recent project, most of my
> C(++) glue functions did very simple things - read some values off the
> Lua stack, call some C++ method, and push the results back onto the
> stack. It seems to me rather wasteful for LuaJIT to setup a stack,
> call my C(++) function, have that call back to the API to read/write
> values, then have LuaJIT tear the stack down again.

Yes, now that everything else has gotten so fast, that's the main
performance bottleneck when calling C functions from compiled Lua
code. And it gets worse: the compiler can't optimize across such
calls, because they might have arbitrary side-effects (changing a
global, growing/reallocating a table etc.).

> For a large
> proportion of these functions, I consider it plausible that LuaJIT
> could disassemble the C(++) function, replace the calls to the Lua API
> with inline assembler, and then be able to call the resulting function
> without having to mess around with moving stuff on and off the stack -
> effectively turning arbitrary C(++) functions into fast functions.

It's plausible, but it won't be pretty. And the complexity grows
quickly. Just think of a C function which calls printf(...) (e.g.
for debugging). You can't easily trace through half of libc to
prove that it doesn't mess with the Lua stack. You're basically
lost at any indirect call (e.g. the PLT).

> Obviously the whole disassembling and re-writing process is more
> complex than the proposed FFI interface in [1], but it would allow the
> re-use of existing binding code

There is already backwards compatibility for existing bindings.
It's just not that fast. If you want it to be fast, use the FFI.
And provided that one can make its use very comfortable, it
shouldn't be too difficult to overcome inertia.

--Mike