Re: LUTask

[skaller <skaller@...>]

On Mon, 2005-01-31 at 02:01, Mark Hamburg wrote:
> on 1/29/05 9:55 PM, skaller at skaller@users.sourceforge.net wrote:
> 
> > [ Described the Felix scheduler ]
> 
> Interesting. In an interactive environment where one had a thread that ran
> for a while perhaps being polite enough to do non-blocking yields but not
> blocking, this would have the issue that it won't revisit the event queue.

Correct. The system does minimal context switching. If there is 
an expensive 'background' thread, it will hog the CPU. There is
no concept of fairness. 

The solution is to use pthreads as well. (pthreads = Posix threads
= pre-emptive threads). Or even processes. 

I guess any Win3.1 or Mac OS <9 will tell you that cooperative
multi-tasking is unreliable in the sense that if even one
'thread' isn't polite, or just plain bugged, the whole system
can lock up.

> On the other hand, the dispatcher could watch a clock or wait for a signal
> from a timer to decide that we should visit the event queue even if the
> running queue is non-empty.

The problem is that the dispatcher is just another coroutine.
It only runs when its minions yield to it. If you use a signal,
you could notice that some minion was a CPU hog, but what can
you do? You can't kill the hogging coroutine. The control
exchange requires the stack is empty. While the thread
is physically running it isn't, and the same is true
inside a signal.

> One further point to note is that if one uses multiple universes to take
> advantage of multiple processors, an inter-universe call is just another
> form of blocking yield.

Yes. The telco system I worked on had to run not only on a 12 CPU Sun
box .. it had to support multiple boxes. This was organised
using a conventional event dispatcher (ACE actually) to partition
the events amoung worker pthreads/processes. We also had a 
distributed asynchonous RT database engine.

The 'threads' were literally one per phone call. Typically,
it didn't matter which CPU or box a 'thread' ran on.

Unfortunately there was one exception. Conference call.
That's where two existing telephone calls get merged
into one call. To do that, the 'threads' had to be mobile,
that is, we had to move one from one process to another
possibly running on another box.

This meant copying the 'stack frame' of the thread across
a network. Note -- not the machine stack, since that
is empty.

> One thing that may be necessary to do to support a Lua scheduler will be to
> replace resume with logic that can look at the results received from a raw
> resume call and decide whether to return them to the caller or to yield them
> up another level. Maybe after I have my morning coffee...

In Felix, there is a 'top level' scheduler. However, any coroutine
scheduled by it can also create coroutines and interact with them,
so you can spawn schedulers hierarchically. You could also organise
to move an fthread (Felix thread) between queues.

But you have to design how you want to do that. In fact,
there is no one universal top level scheduler -- with Felix
at least the client has to write (in C++) their own
scheduler. The compiler just generates libraries of 
functions. The 'flx_run' scheduler runs just one
fthread, thus emulating a stand-alone program.
This one has no event queue.

There is also a 'filter' which feeds standard input
to the client coroutine one line at a time.

There's no event multi-plexing scheduler supplied with
the package, because the design of that is application
dependent. I probably should supply one that can buffer
a set of sockets (schedule based on 'select' or 'poll'
or whatever). The effect of that would be to serialise
socket I/O (amoung multiple fthreads).

A more generic 'top level' for networking would
be nice -- something like LuaSocket. Also I'd like
a GUI scheduler -- one that has one fthread for
every 'window', and schedules based on 'window' events,
where GUI = OSX window or Win32 window or X window
or GTK window .... er .. which is why I haven't
done that one yet .. :)

OK -- all this is a 'design philosophy' discussion.
More specifically and more Lua-centrically: Felix doesn't
provide that top level because it was designed to be
embedded in existing C/C++ architectures. It's middleware.
It fits in between the architectural layer (system control)
and low level data type layer, providing the
'biziness service logic' layer.

So one idea is to embed it in Lua, for simple applications
the top level architecture is written in Lua. The reason being
it is a nice simple language and so for a given architectural
frame work you could knock up half a dozen applications
for a client fairly quickly.

This actually uses the Felix compiler for what it
was designed -- as a *library* generator rather than
a program generator. EG -- you use it to build C level
Lua packages, you use Lua to write the scheduler/
dispatcher (instead of C++).

Felix code still does the real work, not Lua.

Now .. this is what Lua was actually designed for.

As a 'configuration management language'.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net