luajit ffi - faster call to Lua functions from C/C++ code

[S <phd.st.p@...>]

Hi,

First of all thank you Mike for all your work on luajit! :)
I am aware that this point has already been discussed in the thread
"luajit ffi - calling a luajit function via a C function pointer?" but
I would like to add some personal considerations.

I am not strictly speaking a programmer but I have experience with C++
programming in the context of numerical simulations.
It's my understanding that performance is affected in scenarios which
involves repetitive calls of lua simple functions from C/C++ code.

This is partly due to the Lua/C Api and partly due to the difficulties
in optimizing (both from C/C++ and Lua side) this kind of code via
inlining for instance.
I understand that this issue would be solved by porting more code to
Lua (and if I understand correctly calling C from Lua side is not as
expensive).
However, for a number of reasons which have been already discussed
this is not always an option.

My personal point of view is that any performance improvement related
to calling lua function from C/C++ code (accepting that we are dealing
with a sub-optimal situation) would be greatly welcome.
If it's impossible to achieve considerable gains using the existing
Lua/C Api, would it be possible to consider an extension of such
interface to allow for improvements?
I think that something which involves (for repeated calls of the same
functions) only the following sequence of operations to call a lua
function could be achievable:
* dereference a pointer to a C function
* conversion of C types to lua types (for doubles this should be fast)
to pass arguments
* conversion of lua types to C types (same) to return result

The reason I am insisting on this is that I am convinced that there is
a demand for this kind of functionality.
To make a concrete (simplified) example, if you are a researcher in
the context of statistical inference you may come up with a new
inference methodology.
The may quantity of interest which distinguishes between different
models would be the model likelihood: a function from R^N to R.
At the moment, you may use C/C++ to implement your algorithm for a
number of likelihoods corresponding to some model examples.
However the user would have to implement new C/C++ code and recompile
to extend your algorithm to his model of interest and that's usually
not an option.
Usually people use R for this: they interface their C/C++ library
(which has not been coded in R itself for performance reasons) to R
and the user can specify some custom likelihood function in R itself
which will then be an interpreted function.
I can guarantee that this approach is, even now, way slower then
giving the user a lua script to customize for such likelihood
function. Clearly any improvement cannot hurt :)

There are a number of other occasions I can think of where lua can be
used to allow the user to modify parts of the code which is likely to
change (a function to be integrated, coefficients of a stochastic
differential equation, coefficients of a partial differential
equation,...) while retaining good performance.

Thank you for reading :P


Cheers,
Stefano