Re: Thread-support in LUA

[David Jeske <jeske@...>]

On Tue, Sep 09, 1997 at 09:56:55AM -0300, Mark Ian Barlow wrote:
> There's a terminology confusion here, I think. Traditionally "threads" are
> lightweight, and all share the same stack while "processes" are completely
> independent as David describes. Threads do not *have* to share other data,
> however, and current thinking amongst most formal methods folk is that they
> shouldn't be *seen* to (from the user's point of view) even if they actually
> *do* at the implementation level.

I'm talking purely about the mechanism of my implementation. I want to be
able to run multiple copies of the lua environment simultaneously, and in
the same address space. However, I don't want to share "lua data" between
the different lua environments. Essentially, from Lua, it'll just seem
like a normal lua environment, and from "C" it'll seem like you have
several threads (os supported threads) all running, and occasionally
calling into their own private lua environments.

> On this basis I'd say David is on the right track, but Bret has a point: You
> couldn't preemtively multitask between your threads unless they each had a
> protected stack space, because if you did a longjmp() to some point inside
> a previously suspended thread it might subsequently walk all over the call
> stack of the one you just suspended. The only way you could make this work
> using a single stack would be if each thread's stack requirements were
> statically determinable at compile time (with Lua? no chance!).

I'm going to let the OS take care of the threading issues. Yes, each
thread (we're talking real machine-C threads here) will have it's own
stack.

> Pragmatically then, fork/join would be the best way to get things working
> quickly, and pipes could be used for communication. David doesn't say (this
> confuses me a bit) that he wants to multitask his Lua sessions and communicate
> between them but I don't see what use they would be if you couldn't; If you
> just want multiple independent sessions surely you just spawn them from a
> shell script? I suppose this goes beyond ANSI C, however; such a "Parallel
> Lua" would really need POSIX compliance as well. To go the other way would
> need, as Bret says, the splitting off of Lua's call stack from C's and it's
> incorporation into a "Lua_Thread" structure along with the globals.

fork()/join() is not acceptable for what I'm doing. I have a large main
program with lots of data structures. I call into lua to handle "events".
I don't want to have events be purely "sequential", so I want to have
multiple threads, each with it's own independent lua environment. I'll
just call into the private lua environment when an event occurs, and it'll
call back into C to do things, and it'll all be private to this thread.
However, it has to operate on data-structures in my program, so
fork()/pipe() is not acceptable.

In fact what I'm talking about is not radical at all and requires no
specific support from lua. The ONLY thing I need to be able to do with Lua
which I can not currently do is to "create an instance" of the lua
environment. So I can create more instances whenever I want (one per
OS-thread).

> A lot of the basic theory for multitasking derives from C.A.R. Hoare's work
> at Oxford University here in the UK. IMHO, the designers of Java slipped up
> badly when they used his earlier concept of monitors (ie: 'synchronised'
> methods) rather than his later and much superior CSP ("Communicating-
> Sequential-Processes") model as used (but not exclusively) in Ada. In the
> former, a thread-safe application *can* be written, with due discipline;
> in the pure form of the latter it is impossible to write an un-safe one.

I'll handle the multithreadding issues in my parent C/C++ code..

> I'm not suggesting Lua needs strict typing BTW, just that it *doesn't* need
> <yuck> shared memory. I suppose what I *am* saying is that multi-threaded,
> multi-tasking programming is only ever going to be much fun to do on a large
> scale if you have a lot of compiler support, which goes against Lua's spirit
> somewhat.  Java people struggle and thrash around trying to coordinate of the
> order of 10 threads; Ada pushes that envelope a bit, and occam allows me to
> cope easily with 100+ but only at the expense of great formality in the
> language: The pre-compilation checking must be > 80% of the total
> compiler code!

I understand where you're coming frmo, and in principle I agree. However,
I merely want to use OS threads to allow the execution of several
independent "lua events" to run at the same time, so that I don't get a
"serialization" of events. 

The "protected model" for this would involve having a different address
space for each lua environment, which would have some RPC/pipe mechanism
to talk to my mail program. However, this is a game, and it dosn't need
that level of protection because the code involved is tiny, and needs to
be fast. I can't accept the overhead of fork(), and then run lua code, and
then use RPC with the main program just to do the equivilant of:

object[6]:ai_event(); -> (from C)

function ai_event(self)
    /* some computation */
    self:goto(self.newlocx,self.newlocy);
end

-- 
David Jeske (N9LCA) + jeske@chat.net