lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

(This is sort of a free-floating post, since the context comes from other messages.)

The question is: Can you run different (Lua) threads in different OS threads? And the answer is a qualified "yes", but with a great deal of caution. Here are some of the issues:

In theory, you can define lua_lock and lua_unlock and then execute different Lua threads (from the same context) in different (OS) threads. That's what Diego's luathread module does. But there are still a number of issues.

The most important one is that you either have to ensure that each thread has its own globals table, or that you never modify any global, or that you surround all uses of mutable globals with an explicit lock function, presumably implemented by the thread library you're using.

For example, you'd need to replace:

  someglobal = someglobal + 1


  someglobal = someglobal + 1

Or something like that. And then you'd have to worry about the code within the lock throwing an error, an issue which we've been discussing in a different (mailing-list) thread.

You would also have to lock similar expressions with upvalues, if the upvalues could end up being executed in different threads.

Beyond that is the part the Lua script can't control: C modules.

There is only one Registry per Lua context, and some library functions modify the Registry; in particular, the lua_ref mechanism uses the Registry to store references. (That is, it passes LUA_REGISTRYINDEX to luaL_ref.) So any add-on module which uses lua_ref/lua_unref would need to call those functions within a mutex, or the module would have to be recompiled with a different macro definition of lua_ref and friends which called the corresponding lauxlib routines inside of a mutex. (I may be missing something but I don't see anywhere in luathread where this is addressed.) This doesn't affect any standard Lua library but I'm given to understand that a lot of people use lua_ref in their binding code.

The luaL_newmetatable mechanism is also not threadsafe, in the sense that it uses the Registry; I believe the 5.1 package system does that as well. So there may be race conditions associated with dynamically loading modules in different threads.

Finally, although there is not usually much code executed between a lua_lock and a lua_unlock, there is no guarantee. The VM itself is always executed within the lock; the lock is released when a CFunction is called. It is not easy to execute a lot of code without calling some library function or other (particularly as there is an implicit call to 'next' in a 'for k, v in pairs(t)') but it is possible:

  for i = 1, 1e8 do
    j = j + i

will hold the lock for an appreciable amount of time. Also, the garbage collector (or at least some phases of it in 5.1) run within the lock.

The main purpose of running multiple (Lua) threads in multiple (OS) threads would be to allow the threads to communicate, and any such communication would have to be carefully coded to avoid race conditions. On the other hand, if you were writing specific functions in order to allow threads to communicate with each other, you could probably write not much more complicated functions which would allow communication between unrelated lua_State's; the cost of creating a lua_State is not that great, and then you avoid all the overhead of lua_lock/lua_unlock, and the various complications of introducing explicit locking into your Lua code. At least, that's the approach I prefer.