extended dual threading [was: Re: Lua in Parallel System]

["Ashwin Hirschi" <deery@...>]

> > An interesting twist to our scheme is that we actually (can) have 2 threads
> > per Lua state. While many seem to go for symmetrical scenarios I decided for
> > some fearless asymmetry and came up with a "dual-threading" approach, where a
> > controller and worker (OS) thread share one state.

> The dual thread system sounds very interesting. Can you share more details?

Sure. Well, where to start...

First off, the "dual threading" model is event-driven. This means that once a "box" (= Lua state + controller/worker thread pair + message queue) is created, its controller thread waits for a message to arrive in the box queue.

An incoming message acts as an event and triggers the controller thread to call a  (named) Lua handler. That handler can basically do what it pleases. It can inspect the message data and process it itself, or it can decide to pass it on to the worker thread.

If a controller decides to transfer handling to the worker thread, it calls the "worker" function to make its request. This worker (start) function accepts a function and optional parameters, so the controller doesn't need to pass incoming messages on literally, but can do some preprocessing and "control" what the worker does when.

Now, if the worker thread is idling it'll pick the request up immediately (i.e. right after the controller is done). Otherwise the request gets queued (in the workers' queue). At the moment we're simply using a worker queue size of 1. This "one buffered request" approach means we can guarantee a nice worker flow: if the worker is active, it can still accept one request and will handle it immediately after it has completed its current run.

If starting the worker fails (i.e. the worker is active and the control request cannot be buffered) the controller will have to try again later. To enable this scenario, the controller will need to know how the worker is doing. Specifically, it has to be signalled once the worker has finished.

Therefore we implemented a second handler (next to the basic incoming box messages one) for internal "box" events. This handler will receive a "ready" signal every time the worker has finished handling 1 request. That way the controller always has optimal control over what the worker does.

Okay, so that's the basic event model out of the way. On to a couple of threading details...

Of course, the controller and worker threads can never access the Lua state simultaneously. To avoid conflicts, access to the state gets orchestrated using the familiar mutex/event tactic. Both threads will try to obtain the lock and only continue on success.

But the threads of the box pair behave differently once they have gained access. While the controller thread runs unyieldingly, the worker thread sets the linehook before starting its run. In the linehook it'll check a simple flag to see if the controller needs access. And, if true, will release the lock and wait for a signal to continue.

This creates the nice high/low-priority approach of the controller/worker thread pair, I mentioned earlier. The controller always runs uninterrupted [with its stack on top of the worker stack] and at full speed. While the worker will yield on request (i.e. when the controller wants access) by running at a lower speed [under the line hook]. Note also that there's no mutex-locking/unlocking per Lua API call!

Well, there you have it (or at least, part of it). I hope this gives an idea of what's possible and how. If any of the above sounds complicated, I'm not explaining it right. Because, really, it's not [though I admit it took me a few days before I got the details just right... ;-)].

I would also like to stress that all this is made possible simply because Lua is re-entrant. Without the re-entrancy, the controller thread would never be able to take over the state from the worker thread.

Also, remember that this "dual threading" model can be "extended" to any number of "boxes": separate states all have their own controller/worker thread pair and message queue. Using a simple "sendbox" primitive these boxes can run parallel, communicate asynchronously, and thus interact quite easily.

To implement a useful dialog between parts, all that is needed is a basic protocol. And because of Lua's dynamic nature, implementing (or, maybe more importantly, evolving) such a protocol is something that's incredibly easy to do. The "extended dual threading" model takes care of the rest.

I realise there's much more to be said about how things are implemented and why. For instance, boxes are created fairly "dumb", i.e. with fairly empty states and without any dedicated logic. So, a mechanism is needed to ensure the box can be "booted", in order to install the necessary handlers and such. Other housekeeping techniques, like cleanly closing down a box, are needed as well. I'll be happy to elaborate if anyone's interested.

Okay... enough postponing... it's back to work... [:-)]

Ashwin.
--
no signature is a signature.