Re: RES: package system modification (module loaders)

[skaller <skaller@...>]

On Fri, 2005-01-21 at 07:18, Diego Nehab wrote:

> > How? You can't just do a wait loop.
> 
> What are you talking about? You use socket.select to block until there
> is something to do. When there is something to do, you use non-blocking
> I/O to get and send everythying you can. 

Yup, but select() and poll() aren't scalable.
Select is a joke. Poll() looks better but it still uses
an array of sockets. To discover which sockets to read
you'd have to scan the array -- that's O(n) which implies
that the procedure isn't scalable.

To be scalable, you want constant time context switching,
although you might put up with O(log n).

Event based notification is O(1). I saw some talk
of a signal based system for Linux to allow proper
asychronous I/O.

Searching an array of 100 sockets wouldn't be a problem.
However, what if you needed 100K? That's just about
the rating of a commercial telephony switch:
600 call/sec * 3 minutes = 108,000 calls.

[Felix was designed for that environment ..literally
one coroutine per phone call, however the system
provided true event driven notification]

The point is just 'round-robin' of all the waiting
coroutines to check each socket in turn is also
linear .. so poll() doesn't really do anything
in this context except perhaps save some overhead.

It's very nice the Lua system *itself* is event based,
and therefore is scalable, but unfortunate that 
it's based on a C function that isn't.

Note that using poll() or select() there is no need
to do non-blocking I/O -- since you know which sockets
have data to read you can do a blocking read, and predict
it won't block .. :)

> > I'm sure this isn't the only bug in Windows ..
> > if this doesn't work, Windows is screwed --
> > you *have* to be able to cancel a blocked thread.
> > [If you can't you'd have to kill the whole processes]
> 
> Saying you have to be able doesn't make it so. 

True.

> LuaSocket is about
> portability. Most people can't afford not to use Windows, as sad as that
> can be.

OSX looks interesting -- nice desktop for the
illiterate, unix underneath for us nerds .. :)

> ---
> TerminateThread causes the thread to exit unexpectedly. The thread then
> has no chance to execute any user-mode code and its initial stack in not
> deallocated.

[]

Ouch :)

This can probably be fixed (for socket calls like connect() 
which always block) by simply replacing the call with a thunk,
so that the real blocking operation is converted to a non-blocking
one, and the underlying operation done in a special thread you
know enough about to terminate safely. A bit messy,
and a job for a Win32 system programmer not Lua.

I wonder what Cygwin does ..?


-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net