I have done some crude experiments, throwing caution to the wind and simultaneously running "lua_resume()" on a collection of different child states. For simple test functions, it does appear to work fine; and as I said earlier, I'm starting to suspect that there might be some relatively simple changes you could make in the guts of the interpreter that would make the approach more or less safe even while running more complex coroutines in parallel.
Just for the fun of it, I did take a shot at hacking together a thread-safe lua variant the other day. My approach was pretty simple, I used Slim Read/Writer locks to put optional shared-mode read locks or exclusive mode write locks around all table accesses. Each table had it's own mutex, which increased GCObject sizes, but, I only triggered the locking logic if I knew I was running in a multithreaded mode. Then I stuck another mutex in the global state, and used that to serialize the various newobj calls (but, again, only triggering the locking logic if I knew I was executing in a multithreaded mode).
It wasn't that hard to code up; but, it didn't really work very well. The first problem is that my own application involves a /lot/ of tables, so even being very conservative, and using a light weight locking mechanism that was if-ed out most of the time was still enough code/memory bloat to increase the runtimes of serial code executions by nearly 50%. And when I turned those locks on, and ran my threads in parallel, it appears that the cost of the locking logic meant I lost more than I gained in runtime performance -- I could keep all 8 of my virtual cores busy, but, my multithreaded executions times where, if anything, slightly slower than just turning the locking logic off and running everything in serial.
My sense is that a multithreaded lua hack might start to make sense on machines with 16+ cores, or perhaps in an application where table reads and writes form a smaller chunk of the total operations. But, in my own use case, it looks like a dead end.
In future, I'll be returning to my old mantra: "If it's not fast enough in Lua, rewrite it in C."
-Sven
