[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Feature request: more integration of Lua and C memory management
- From: Jamie Webb <j@...>
- Date: Tue, 8 Jun 2004 22:51:07 +0100
Many libraries need to hold references to Lua objects, and to expose userdatas
to Lua code, and this creates a nightmare for memory management. Every object
must be either under the control of the Lua GC, or managed by the C code.
Consider some struct, which is heavily used and stored in both C and Lua
code, and passed between the two often. There are two options:
- Manage the struct from C.
AFAICT, the only way to be sure that both Lua and C no longer contain
references to an instance is to maintain a reference count (even if that's
just a pair of flags indicating whether the object has Lua and C references).
This appears to be necessary regardless of the memory management strategy
used on the C side. There is then a possibility of an uncollectable reference
cycle between C and Lua, which would be very expensive to detect. Requiring
Lua users to avoid reference cycles would be onerous, bad practice, and
potentially slow. I am particularly concered about reference cycles because I
think that when closures are being passed to C code, they are almost
inevitable, and they could potentially involve a large number of objects.
- Create the struct as a userdata.
Now everything is fine from the Lua side, but every time a reference is kept
on the C side, it must be stored in a table that will be GCed along with the
reference holder (which must itself be a userdata). That means having a
unique metatable for each userdata (I don't think any scheme with weak tables
will work correctly, and it would be too slow anyway). This though makes
object creation too expensive (I'm expecting lots of little short-lived
objects, and making them userdatas means I can't pool them). Also, this may
require additional bookkeeping to link pointers back to userdatas. I don't
believe any other solution (e.g. luaL_ref) can collect reference cycles.
The only solution that I can see is to improve how userdatas hold references
to other Lua objects. Probably the neatest way is to associate with each
userdata an array (length specified when the userdata is created) of
TObjects, in which it may store Lua objects, and which the garbage collector
knows about. That array should be accessible given only a pointer to the
userdata, and not the actual Lua object (e.g. by storing the array
immediately before the userdata and counting backwards).
What I'd really like to see though is more options, to make it possible to
find solutions to these sorts of problems that are both fast and correct. At
the moment, Lua memory management seems very much 'off limits'. Examples
include allowing a userdata to be returned to a pool rather than free'd when
it is GCed, and allowing C objects to take part in marking. C code shouldn't
be reduced to the efficiency of a scripting language just because it
exchanges objects with one.
Some of the isues I've mentioned here have been described before by e.g. Rici
Lake and Mark Hamburg, but there doesn't appear to have been any resolution,
or any input from the Lua authors.
-- Jamie Webb