Re: Lua C API and 1:1 lua_state : os threads

[William Ahern <william@...>]

On Mon, Mar 10, 2014 at 10:50:53AM -0500, Andrew Starks wrote:
<snip>
> So, we've had to do all of the things that others do as well; moving
> Lua values between states being the most obvious example. We are using
> a message passing pattern, of course, and are very happy with how Lua
> is working out, so far.
<snip>
> Does anything come to mind when one thinks of improving the C API, Lua
> and managing multiple states between operating system threads[2]?
> 
> --Andrew
> 
> [1] We thought about a multi-state xmove that worked across lua_states
> and that didn't support lua functions or c closures. But then we
> thought that might be better as a separate library...

One issue with moving objects between different global states is handling
panics and rewinding the locking state. In my project (cqueues) I use
message passing over sockets for most communication. But on thread creation
I need to copy the startup routine and arguments.

I have a carefully choreographed (and documented) block of code to do the
correct thing and safely return the error to the caller. But I first
serialize the arguments before passing them off to the new thread, so the
only panic or error I have to deal with is from the new VM.

If you try to write xmove for separate global states, and especially if you
implement table copying (without serialization), then you'd have to worry*
about panics and errors in both the source and destination VMs. That gets
really hairy when you toss OS thread synchronization into the mix.


* Well, I guess you're not required to worry about it. AFAICT, Lua Lanes
just punts the issue. But in complex environments with complex code, it's
not uncommon to hit memory limits (e.g. via process ulimit), or having to
deal with tables with metatable routines that might blow up inside the copy
operation.