Re: Coroutines and blocking IO

[Jon Smirl <jonsmirl@...>]

On 1/26/06, Glenn Maynard <glenn@zewt.org> wrote:
> FWIW, I don't think this works with disk files on traditional Linux kernels;
> select() always returns them, and I/O always blocks.  Blocking calls to
> read a file typically wait more briefly, of course, but they do block,
> resulting in unused time if no other threads are running.  If you're reading
> from, say, a CD-ROM, seeks may cause much longer blocking.

You want to open the files with O_NONBLOCK and assume that they will
block in the lua code. Local file systems may ignore O_NONBLOCK and
briefly block you in the kernel. But what if the file is on a network
device? In that case O_NONBLOCK will be implemented since it may take
a while for the file IO to complete.

> implement read()=EAGAIN, call out to a registered callback if present,
> epoll_wait(), get event, return.

Note that if the filesystem doesn't implement O_NONBLOCK you simply
won't get the EAGAIN return from the read call. If you code assuming
O_NONBLOCK is implemented the same code will work if it is not.

lua should definitely provide a run-time implementation of sendfile()
which uses the underlying OS implementation (they all have it).
sendfile() is an example of something that definitely provides the
EAGAIN, non-blocking return.

Think about the problem of a 56K dialup modem asking for a 2MB file
from a lua web server. It is going to to take eight minutes to send
the file to the modem. Obviously async IO has to be used or the web
server will be useless. If you set both the net and disk fd to
O_NONBLOCK and then call sendfile() you will get a stream of
EAGAIN/epoll events that will allow you to multitask. Apache,
lighttpd, etc all do this.

--
Jon Smirl
jonsmirl@gmail.com