Re: luajit ffi - calling a luajit function via a C function pointer?

[Paul Bonser <misterpib@...>]

On Thu, Feb 10, 2011 at 6:45 PM, Mike Pall <mikelu-1102@mike.de> wrote:
> Paul Bonser wrote:
>> On Thu, Feb 10, 2011 at 12:51 PM, Geoff Leyland
>> <geoff_leyland@fastmail.fm> wrote:
>> > You're right, someone *could* rewrite ipopt in Lua, but, for
>> > me at least, that seems about as hard as rewriting LuaJIT
>> > would be, and anyway, maybe next week I'll want to use couenne
>> > [2].  I don't expect any magic from LuaJIT to help in this
>> > situation, but I think it does occur legitimately.
>>
>> If I understand correctly, someone could use ipopt from within LuaJIT,
>> with callbacks written in Lua (as soon as Mike finishes adding
>> callbacks to the FFI), and then they would get close to the same
>> performance if the entire thing was written in C, right?
>
> Nope, the performance wouldn't be close to C in this case. :-/
>
> Lua is a dynamic language and the JIT compiler extensively relies
> on the ability to move all of the dynamic overhead out of inner
> loops. This works fine if your code is all in Lua, because
> eventually it _will_ find a loop for any code that runs more than
> a few nanoseconds.
>
> The dynamic type checks and specialization checks only need to be
> done once, and the inner loop stays clear of that overhead. All of
> this relies on the fact that the JIT compiler has complete control
> over the execution, because it stays in pure Lua code for the
> entire duration of the loop.
>
> But this doesn't work with a short linear piece of Lua code that's
> called from C and immediately returns to C. All of the context is
> lost between repeated calls and the state of the universe may have
> completely changed as far as the Lua side is concerned.
>
> So all of those type checks and specializations have to be redone
> _on every call_. The generated code would be faster than the Lua
> interpreter, but certainly not by several orders of magnitude.
>
>> Also, in Shawn's case, if he could call into his C/C++ main loop using
>> the FFI, after also using the (as yet unimplemented) callback
>> functionality to register his Lua callbacks with the C/C++ code, then
>> he would also reap the benefits of having his lua code JITed?
>
> Same problem as above. All context is lost between calls. The
> C side suffers, too. But not as much as the Lua side, since it's
> statically typed and it _is_ the caller and can keep context,
> which means it has far better optimization opportunities.
>
> --Mike
>
>

Well dang. I guess that was a little too much magic to hope for :)

Either way, I'm still looking forward to the callback functionality
because it will mean that I can write entire applications which make
use of C libraries without writing a line of C, even if the library
provides its own main loop.

It'll be good to keep in mind that ideally I should try to write my
own main loops, though.

-- 
Paul Bonser
http://probablyprogramming.com