Re: LuaJIT 2 ffi (casting / force hotpath)

[Florian Weimer <fw@...>]

* Mike Pall:

> Florian Weimer wrote:
>> Is there a way to mark af FFI calls which may trigger callbacks (via C
>> extensions)?  What's the general story about callbacks?
>
> Right now the call chain 'FFI -> C -> lua_*()' with the same
> lua_State is a big no-no. I'm planning to add an extra GCC-like
> attribute that could be used to mark C functions which do that.
> So you can still call these functions from Lua, but it's only done
> from the interpreter (e.g. for the entry into a GUI main loop).

Would the jit.off(func) approach work right now?

Could you bail out the interpreter only if a callback actually
happens, or would that overhead apply in all cases?

> [You may however call luaL_newstate() from the FFI to start up an
> _independent_ Lua state and play with it. Works fine.]

Interesting idea.  However, for things which are as intertwined as
SQLite or Curl, this wouldn't work. 8-(

> Converting a Lua function into a callback for a C function is a
> different issue (the canonical example would be qsort()).

There is qsort_r now (with non-portable argument ordering, however).

> This is quite tricky and has some inherent issues, e.g. dynamically
> generated trampolines cannot be deallocated. So I've postponed this
> for now.

The usual void * parameter makes the per-closure trampoline
superfluous.  Would it be possible to put the auxiliary data into some
special cdata object, and require that the programmer keeps the object
live while the callback can be called?

> Another option would be to add a ffi.osdef convenience module,
> which has all of the generic POSIX types, or the LPCTSTR mess for
> Windows. The latter wouldn't be too bad, because the Windows types
> are cast in stone. But the former requires tricky pre-processing
> to get the needed OS-specific types without including lots of
> extra cruft.

I have done this to generate Ada bindings.  Currently, if you know
that something is an integral type, you only need to check the value
of sizeof and the signedness of the type.  This gives you enough
information to reconstruct an equivalent typedef.  Errno constants can
be extracted using brittle GCC hacks such as "gcc -E -dN -x c
/usr/include/errno.h | grep '^#define E'", and generating a C program
from that which prints their values.

> So I've thought about adding an __errno attribute. If you declare
> a C function with this attribute, you get the errno as an extra
> return value (local ok, errno = ffi.C.mkdir(...)). This is fetched
> by low-level code immediately after the function returns. Alas,
> this means I have to figure out all of the variants for getting
> the errno value myself and teach them to the JIT compiler. :-/

For my Ada stuff, I called a get_errno function like this one:

  #include <errno.h>
  int
  get_errno(void)
  {
    return errno;
  }

(Apparently, Oracle still ships code with "extern int errno;" in it,
which makes things much easier, but puts the affected software firmly
into the past.)