Re: RES: package system modification (module loaders)

[skaller <skaller@...>]

On Thu, 2005-01-20 at 18:44, Diego Nehab wrote:
> Hi,
> 
> The posibility of a library or something else requiring modification to
> the Lua core or to the interpreter executable has nothing to do with the
> package proposal.

Sure it does -- packages are designed to help load
extensions in a standard way.

The fact is that some extensions need to extend the
core an others don't -- but the client doesn't care.

> I have no idea of how Python allows changes to the core interpreter
> automatically, but I am listening. 

It doesn't. The core was changed over time, many times,
to allow extensions like signal handling and threads
to work. Threading fits in with Python better than Lua
because the interpreter does virtual threading anyhow,
pthreads work by blocking pre-emptions except when
the interpreter would have switched threads anyhow.
Ocaml bytecode is the same. 

Lua is actually better, IMHO, because it doesn't
support threading in any fixed way .. but that also
makes it harder to add thread support.

> > To do sockets, you basically have to have threads
> > to handle blocking/cancellation, and that intervenes
> > in the core too if you do it right (so coroutines never
> > block).
> 
> Not true. You should use non-blocking I/O and avoid blocking altogether.

How? You can't just do a wait loop. What you really want
is blocking, but only blocking the current 'thread'.
In particular, you may want to 'yield' from a coroutine
which is resumed only when the request would be satisfied.

This isn't so easy to organise. (Ocaml bytecode interpreter
does it though).

> I don't think you can cancel a blocked thread safely. 

On the contrary, that is precisely where you DO cancel
Posix threads -- they're called cancallation points,
and of course they're always blocking operations.
A non-blocked thread can poll for a request to die,
a blocked one cannot. One typically wants to cancel
a thread precisely *because* it is blocked.

So basically a signal to kill a thread will
cancel it when it is blocked .. when it isn't,
the signal handler can run and notify the running
thread instead.

> On Windows at
> least I read somewhere that it can be catastrophic...

I'm sure this isn't the only bug in Windows .. 
if this doesn't work, Windows is screwed -- 
you *have* to be able to cancel a blocked thread.
[If you can't you'd have to kill the whole processes]

> > > LuaSocket, for instance, requires no such rebuild. It's a pure add-on lib
> > > in C and Lua.
> >
> > And therefore probably doesn't work properly with threads
> > and coroutines.
> 
> Not true either. Works fine with coroutines. 

Of course it doesn't.  A blocking operation, such as a read,
will block that coroutine, and all others. No good.

A non-blocking read will make the system spin, so that isn't
a solution either. What you actually need is the equivalent of
the Felix primitive 'read x', which does a blocking read
by yielding (in the same operation).

That is, you need to *schedule* coroutines that block,
so they're resumed when blocking operations will not block.

In other words you need to do just what an operating
system does -- allow the user to exchange control
to the system with a read request, so the system
can continue and resume the coroutine when it's
request won't block.

This is how Felix coroutines/mircothreads are intended 
to work. However 'read x' isn't limited to socket operations,
in fact I haven't even implemented them. Instead it is intended
to work with an arbitrary message source, for example GUI
messages.

Thus, you have a coroutine for every widget, and you 
read a button push. There's no callback in your code.
The actual button push callback from the GUI resumes
your coroutine. Something like:

do forever
	e = read_event()
  	if e == Press then 
		display_depressed()
		e = read_event()
		if e == Release then
 			display_released()
			notify_clients ("Click")
		end
	end
done

This requries an event dispatcher: 'read_event()' actually
yields control and is resumed with the next event for
that widget -- read_event() is a blocking operation,
but it doesn't block it yields. 

Otherwise your GUI would freeze up, since you have
these coroutines for every widget, and you don't
know which event the client will trigger next.

So you'd do something like:

do forever
	e = read_gui_event()
	w = widget_number_of_event (e)
	thread = widgets[w]
	resume (thread,e)
end

Socket events would be the same.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net