Re: Coroutines and blocking IO

[Glenn Maynard <glenn@...>]

On Thu, Jan 26, 2006 at 04:19:43PM -0500, Jon Smirl wrote:
> You want to open the files with O_NONBLOCK and assume that they will
> block in the lua code. Local file systems may ignore O_NONBLOCK and
> briefly block you in the kernel. But what if the file is on a network
> device? In that case O_NONBLOCK will be implemented since it may take
> a while for the file IO to complete.

But it won't be.  Give it a try; reading files over Samba will block on
Linux just as much as a local read.  (Havn't tried NFS.)  And reading
from a CDROM can be even slower than reading files over a network, and
that'll block, too.

The point is just that if you want to implement an API for yielding during
blocking calls by transparently turning them into nonblocking calls,
it's either going to take nonportable implementations, threads, or have
a rider: "this may not work with all types of files".

> Think about the problem of a 56K dialup modem asking for a 2MB file
> from a lua web server. It is going to to take eight minutes to send
> the file to the modem. Obviously async IO has to be used or the web
> server will be useless. If you set both the net and disk fd to
> O_NONBLOCK and then call sendfile() you will get a stream of
> EAGAIN/epoll events that will allow you to multitask. Apache,
> lighttpd, etc all do this.

But you won't: O_NONBLOCK is almost never implemented for disk I/O.  (It's
always implemented for any respectable sockets library, though.)

In web servers, this is more commonly done by having a separate thread
for each file.  For example, if you send a file off of a CDROM with
Apache, even if it's thrashing the disc due to competition with other
processes, it won't cause other users to be blocked by it.  This
wouldn't work with a nonblocking implementation in Linux, since those
calls aren't actually nonblocking for CDROMs.

-- 
Glenn Maynard