Re: Coroutines and blocking IO

[Javier Guerra <javier@...>]

On Thursday 26 January 2006 5:06 pm, Jon Smirl wrote:
> You can have APIs that block without supporting a non-blocking API
> still keep lua working. Down inside the C implementation of the
> library create a native IO thread for the problem subsystem.  The IO
> thread goes off and does the blocking IO leaving the main thread to
> keep running lua code. When the non-blocking IO completes the IO
> thread messages the main thread. On the next lua schedule it picks up
> the completed IO from the IO thread and returns it to the lua code.
>
> Just because libraries are using IO threads doesn't imply that lua is
> aware that native threading is happening.

Apple does a similar trick for the classic-like asyncIO in Carbon libraries 
over the blocking (but multithreading) BSD kernel in MacOS X.

I also managed to implement something analogous for Copas.  the idea is that 
in a cooperative environment like this, you should yield often, but what if 
you have to use a blocking library?  my solution looked like this:

function handler (xxx)
  -- standard copas thread
  -- be careful not to block!
  helper.bg (function ()
    -- 'background' thread!
    -- do whatever you want!
  end)
  -- back to cooperative
end

it used LuaThreads for creating a new preemptive thread and execute that 
anonymous function, meanwhile, the copas thread yielded and everything else 
kept going.  when the background thread was finished, the copas thread was 
unblocked and eventually resumed.

it worked... barely.  hugely unstable, after 4 to 9 threads, it just died.

it remembered me of something:

" since it's twice as difficult to debug code than to write it, you should 
never write at your highest capacity, because then you wouldn't be able to 
debug it "

-- 
Javier

Attachment: pgpT9cHHNv4zo.pgp
Description: PGP signature