Re: About coroutines and light threads

["Sebby " <beast@...>]

Hello.

> Hi,
> 
> Sebby wrote:
> > 
> > After checking out what coroutines are, it seems very similar to 
the
> > approach that was suggested (and that i tried) using 
setjmp/longjmp.
> > From what i noticed, this approach doesn't really require any
> > modifications to lua (one of the big advantages). But then again,
> > since no modifications are requiered inside lua, why not just 
make a
> > coroutine library and give some examples on how to use them with 
lua?
> 
> Lua has to be changed.  Each coroutine needs a local C _and_ Lua 
stack.
> So, you do not have one Lua stack but as many as you have 
coroutines.
> And these additional stacks have to be taken care off by the garbage
> collector.  (And, coroutines should be garbage collected, too.)

Ok. Well as for having a different stack per lua thread, it's 
possible without actualy modifying lua. I have basicly wrote an API 
that sits on top of lua and takes care of different tasks. For 
example, it will create lua_states for different threads, and will 
even bind the global section with the thread specific section when 
comes the time of execution. Other thing it will take care of is the 
general scheduling of the threads, allocating their C stack and 
providing a "start" function for each thread. The only hack required 
for this is copy/pasting the code from lua_open to create your own 
thread specific states.

As for garbage collection, well. Do you really need to collect? I 
mean, when a thread is done executing, it's stack is useless and 
should not contain any information. So you could technicaly delete 
it. Worst case, you could always send it to lua as a UserData with a 
specific tag and just set the GC tag method for that tag. This way, 
the garbage collection will be automatic without any lua 
modifications.

> > Out of curiosity, how would you do coroutines without 
setjmp/longjmp?
> > Basicly copy the state of the CPU (IP, Registers, Stack 
Pointer,...)
> > to a structure and restore it? If so, why not simply use
> > setjmp/longjmp since it basicly does that. In my test 
implementation,
> > the only line of assembly code required was to set the stack 
pointer
> > to the coroutine(or thread) specific one.
> 
> And this is where sigaltstack helps.  sigaltstack allows you to set
> a stack used for signal handlers.  You just set one and issue a 
signal
> to yourself.  That way the kernel does the "load esp,stack" for you.
> Now you can perform a setjump and you have a jmpbuf with a stack 
pointer
> to a different stack.

Hmm. I wonder how standard sigaltstack is. I'll have to look it up in 
visual studio tonight. If it's standard then it should be possible to 
write a completeley platform independant solution.

> 
> > The biggest problem with the coroutine (or setjmp) approach, is 
that
> > you have to allocate a stakc per thread and have to find an
> > apropriate size what will allow all threads to run without 
overflow
> > yet be carefull not to overuse memory.
> 
> That's a limitation of the standard implementation of C.  But I 
don't
> see this as a problem.  Each thread library has to make this 
decision.
> At least under Unix like systems you just create a pretty big stack
> (iirc Linux pthreads use 2MByte per thread).  You just have to know,
> that memory isn't really allocated.  It's only allocated if you 
actually
> touch it.
>   Btw, it's the same with the regular C stack.  It has a virtual 
size
> of 128MByte or so.  It's the behavior to defer memory allocation to
> the moment you really need it that makes this possible.
> 

Well on a PC running windows, you cannot simply assume you have 
plenty of memory allocated for a stack and simply allocate it on 
demand. However there is way to reserve chuncks of memory via the OS. 
The problem is that on other plateforms like Playstation2 which only 
has 16MB of memory, the allocation has to be considered carefully 
since you'd want to use the most of that memory for actual 
geometry/textures rather than thread stacks.

Sebastien St-Laurent
Software Engineer, Z-Axis ltd.
sebby@z-axis.com