Lanes 2008 revise

[Asko Kauppi <askok@...>]

The doc is _mostly_ on par but unfortunately the change log was  
lagging behind.
Here's a better one:

	• lanes.new (renamed from lanes.prepare)
	• h:unpack() (renamed from h:results)
	• Lindas for all inter-thread communications (replaces FIFOs, tubes,  
thread groups)
	• Fast inter-state copy using hidden "keeper states" (replaces  
serialization)
	• Also closures can be copied between states (yes, upvalues copied by  
value)
	• Full userdata is no longer copied as light userdata between states;  
instead
	  a new concept of "shared userdata" can be used to make userdata  
that is
	  passable between states and who's lifespan can exceed that of the  
creating state.
	• 'error' field for catching a lane's possible error


The changes are BIG. Basically, only the underlying threading part (OS  
specific things) has remained fairly intact. Everything about the  
internals is redone, in good Lua remaking fashion (if you can live  
without it, throw it away).

Lindas provide a wealth of opportunities for any kinds of inter-state  
communication needs. Now, what I'd very much like, is to have a group  
of interested people help me fine tune the API's, before we make this  
truly public. Having a go with Lanes in some real world projects would  
soon show places where the API is suboptimal, and needs for features  
that are not currently there.

Like the first ones:

Watchdog.  Will make a sample on how to do it, ms-level resolution but  
based on wall clock time, not CPU time.  I am not sure if there's a  
way to get per-thread CPU statistics, anyways.

Limiting thread number.  No, you currently cannot, but it would be  
easy to make a limit. API suggestion, please? ;)

You can limit the libraries a lane has available. If you don't allow  
'package' (require), a lane won't be able to spawn sublanes.  But  
that's probably not the optimal solution?

More than one Lua state per OS thread.  Currently, they go 1:1, and  
you should use coroutines for multithreading within the one state.

Reason for this is at least pending operations (Lindas), where I don't  
see a way to share the OS thread with multiple Lua executions. Please  
share ideas if you feel different.

Then again, threads should be light-weight entities on any OS, and we  
can succesfully run up to 1000 threads or more (memory is the limit)  
already with older Lanes (2007).

-asko


Alexander Gladysh kirjoitti 19.7.2008 kello 19:47:

> On Sat, Jul 19, 2008 at 1:58 PM,  <askok@dnainternet.net> wrote:
> > If (as I think) you're talking about OS level pre-emptive threads  
> > (and not
> > Lua coroutines), I would strongly suggest looking for a solution  
> > s.a. Lanes
> > for this.
>
> Yes, I'm talking about OS threads.
>
> > I've just finished work on a full revise of it, and will make a  
> > fine tuned
> > release early next week.  It will give you plenty of tools to get the
> > cancellation done what you're looking for.
>
> Great. Is there a changelist? (If not, I can wait until release,
> nothing urgent here.)
>
> > By the way, would you rather limit the amount of time consumed
> > by the Lua state (a true watchdog) or just the number of Lua  
> > instructions?
>
> As to my current situation -- I can settle for number of instructions.
> I simply need to catch runaway user scripts. On to the other hand, a
> true watchdog would be useful (not necessary with less-than-second
> granularity) -- think, for, example, of analogue to php.ini
> max_execution_time variable.
>
> > Early prerelease is available in LuaForge. 'make test' works on  
> > Linux and OS X.
>
> You mean here: http://luaforge.net/projects/lanes/ ?
>
> Am I correct that documentation in the archive (index.html) was not
> yet updated for the new version?
>
> Also, a question (sorry, didn't find the answer by skimming the docs).
> Would I be able to limit the number of threads (and, possibly, to
> allow more than one state per thread, with cross-thread job-stealing
> -- but that's wishful thinking)?
>
> Alexander.