lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

----- Original Message -----
From: Mark Hamburg
Date: 1/11/2011 9:37 AM
What's wrong with references? Basically, the issue is that they are dangerous in a number of potentially surprising ways. For example:

* They create false matches in multi-universe contexts. Since they are just allocated as integers within a universe, there is no good way to tell that a reference isn't being used in the right universe. It will still likely lead to an object, just the wrong object. Using a light userdata as a key means that you will end up at nil if you try to look for the matching Lua object in the wrong universe. (This could also be addressed by using a global, atomically incremented counter for generating reference id's rather than the existing reference mechanism.)
I think someone else already commented on your point here, but I don't buy into this. *You* are responsible for knowing which integer goes with which 'universe' (is that even really a Lua term? I assume you are referring to each lua_newstate() call). In any case, I don't see it as being any different than holding onto a stack index in a C variable and using it irresponsibly within another Lua 'universe'.
* If you use the patch to add a global-interpreter-lock to Lua, it won't actually work properly with the reference logic because the reference logic lives on the wrong side of the GIL. If you try to create two references at once, you will probably foul the reference system.
This part is true. The current luaL_ref() setup will not work properly with the lua_lock()/lua_unlock() patches, but the techniques I describe below should work fine with a GIL.

I use Lua for a lot of data. I rely on mechanisms that work fast to process that data. That's why I created LuaPlus' LuaObject, a class that embeds the TValue information right in the object and allows access and manipulation as quickly as possible. Without modification of the Lua core and not exposing any internal data structures (which kills me with LuaJIT), I have another version of LuaObject that uses the *much* slower luaL_ref()/luaL_unref(). Being unsatisfied with the performance there, I resurrected the old Lua 4.1 references and found the use of those put performance on par with LuaObject with the embedded TValue information for many important operations.

All code discussed below is available in the 'fastrefs' and 'c41fastrefs' branches of the Git repository at git:// You can visit the changes made for the 'fastrefs' branch at The c41fastrefs changes are found at

First, some contrived benchmarks from my Core i7 720 laptop.

c41fastref: 0.111000
c41fastref-oneloop: 0.097000
lua_fastref: 0.119000
lua_fastref-oneloop: 0.111000
lightuserdata: 0.212000
lightuserdata-oneloop: 0.210000
luaL_ref: 0.402000
luaL_ref-oneloop: 0.405000

These represent the lowest timings that came through for the benchmark in seconds. Initial passes are somewhat slower due to memory allocation considerations, although when I ship a commercial game title, I always ensure the proper amount of memory is allocated up front.

What do we learn from this?

* c41fastref is a derivative of the all C Lua 4.1 reference system. Reference integers retrieved from lua_c41fastref() can be passed to any Lua API accepting a stack index, speeding up operations and making the reference system more user friendly. The C data structures here are the *fastest*. * lua_fastref is based on the current luaL_ref system, but it exists in lapi.c and accesses data structures directly. Reference integers retrieved from lua_fastref() can be passed to any Lua API accepting a stack index, speeding up operations and making the reference system more user friendly. This one is close enough to the c41fastref performance that I can reasonably use this technique without any real worry about performance loss. * lightuserdata is a technique eluded to within this thread where lightuserdata is used to key the storage. Although only twice as slow as 'c41fastref', the use of lightuserdata uses *twice as much memory as the 'c41fastref', 'lua_fastref', or 'luaL_ref' techniques*. * luaL_ref is the current Lua 5.2 reference system. It is 4 times as slow as 'c41fastref' and blows the CPU cache more easily.

Nevertheless, simply removing the luaL_ref system is not the answer, as anyone needing such a system will have to invent their own. Their own inventions will more than likely be slower or use more memory or not work properly with the garbage collector. I believe the only current problem with the luaL_ref() setup is in a multithreaded environment due to the lua_lock()/lua_unlock() combos being useless within the luaL_ref()/luaL_unref() APIs. That is fixable, and then, at least, there is a working system for users.