Re: Storing table references in CDATA

[Mike Pall <mikelu-1205@...>]

Arran Cudbard-Bell wrote:
> The string key and the return value are easy to represent in
> cdata, but I can't figure out if it's possible to get or store
> pointers to lua objects. GC isn't a problem because the tables
> are referenced in other places.

Well, that's not gonna fly. Either go all the way and use cdata
everywhere (except for rooting it somewhere in a table) or don't.
You can't anchor, nor store, nor retrieve GC objects from cdata
objects.

[One reason is that the compiler makes use of its knowledge about
separate object domains for optimizations. E.g. storing something
to a cdata object cannot possibly affect the contents of a Lua
table. This in turn means a load from a hash table can be hoisted
out of a loop across arbitrary cdata accesses and even across
C calls performed via the FFI. Not so with the classic Lua/C API,
where all guarantees are off.]

> I've not tried embedding LUA in another application yet, so
> maybe what i'm looking for is somewhere in the C API, if this is
> the case then i'll gladly go RTFM, just want to know if it's
> possible?

You'll certainly want to stay away from the classic Lua/C API as
far as possible, if you want performance from LuaJIT. Try to use
pure Lua code everywhere as much as possible. Call C functions
only via the FFI, but never via the classic Lua/C API. Also, avoid
callbacks from C at all cost. And only resort to C when it's an
existing API, but don't try to prematurely convert something to
C code, because you think it might be slow in Lua. And/or undo
such conversions, in case you made them for the Lua interpreter.

> * Slow is sort of relative here, the system can do about 130K
> TPS traversing 5 nodes, and inserting 5 fields (yey LuaJIT), but
> were looking for something in the range of 500K. Almost
> certainly going to have to move to using CDATA instead of lua
> tables at some point, but thats more complex, and this is more
> of a POC currently.

Judging from past experience with these kind of integration
projects, I guess your performance is most likely dominated by API
friction (callbacks from C, marshalling to C calls) and not by the
data structures themselves. Check the compiler output with -jv or
-jdump or the equivalent Lua code: require("jit.dump").start()

--Mike