coroutines and yield / preempting in linehook

[Brian Swetland <swetland@...>]

I did a little experimentation to confirm my suspicions about lua_yield()
and (unfortunately) confirmed what I suspected...

First, thank you Tom Wrensch, for your example scheduler for coroutines
(http://lua-users.org/lists/lua-l/2002-06/msg00099.html), which worked
as a great testbed (just needed to change coroutine.create() to 
coroutine.wrap() and it worked fine under lua 5.0 beta).

So, you *can* set a linehook that is a C function that calls lua_yield()
and a yield will happen whenever the opcode count rolls under (as
expected).  You must set this linehook on the lua_State of the thread
you want to be prempted this way -- I exposed a function written in C
to do this called "setpreemptcount" and away I went.  I then had two
threads merrily printing "task a" and "task b", switching between each
other every 50 opcodes.

Unfortunately, if the hook triggers while a metamethod is running
(or any other case where lua calls c which calls lua), I get the
"attempt to yield across metamethod/C-call boundary" error.

Since doing anything interesting in lua tends to require metamethods
(and possibly needs c->lua callbacks), the "yield in linehook" technique
is not very useful in real programs unless I'm missing something obvious.

Has the lua_newthread() and lua_closethread() been used in actual
threaded environments?  I gather I need to provide lua_lock() and
lua_unlock() macros in lstate.h to provide critical section protection.
Are there other gotchas?  

It looks relatively pricy in that, from what I can see, the vm has to
lock around table access, stack writes, etc.

...

In case anyone cares, my goal here is a little server framework where
each connected client can be handled by a separate lua thread.  I have
something similar where I use c->lua callbacks to handle data in and
data out, and that does work, but it doesn't prevent one 'process' from
starving others.  

Also, happy holidays everyone and thanks for this great language :-)

Brian