Re: on the cost of non-blocking LuaSockets

[Valerio Schiavoni <valerio.schiavoni@...>]

Thanks to everyone for the interesting insights on the thundering herd issue, I surely learned more than expected from this discussion.

However, i still have the doubt that there is an inherent performance issue when using non-blocking sockets attached to co-routines that need to read big chunks of data (hundreds of MB). 

Do I understand correctly: the fact that the coroutines need to periodically yield/resume imply that the achievable download throughput is necessary reduced ? 

This seems like a big issue to me...

 PS: I agree with Sean Conner, Networking is really hard

On Fri, Dec 12, 2014 at 8:06 AM, Sean Conner <sean@conman.org> wrote:

It was thus said that the Great William Ahern once stated:
> On Thu, Dec 11, 2014 at 06:11:00PM -0500, Sean Conner wrote:
> > It was thus said that the Great William Ahern once stated:
> > > On Thu, Dec 11, 2014 at 12:31:28PM -0500, Sean Conner wrote:
> > > > It was thus said that the Great Valerio Schiavoni once stated:
> > > > >
> > > > > Due to continuous 'timeout' errors returned by the receiving socket (which
> > > > > is set to nonblocking, socket:settimeout(0)), the coroutine continuosly
> > > > > yields: in this case, is the reception of the data somehow put 'on hold' ?
> > > >
> > > >   You don't need to set the socket to non-blocking if you use select just
> > > > for reading (and I just checked our codebase at work---I don't set any of
> > > > the sockets to nonblocking).  All select() does is indicate when a file
> > > > descriptor (in your case, a network socket) has received data (ready for
> > > > reading), can write data without blocking (ready for writing [1]) or an
> > > > error happened.
> > >
> > > This is not true at all, and people who assume this are running code with
> > > TOCTTOU bugs. Spurious wakeups can be very common. There are two common
> > > scenarios.
> > >
> > > 1) Two threads reading from the same socket. Both get woken up, one drains
> > > the socket, the other ends up stalled. This is easy to avoid, but proves the
> > > point about the semantics of readiness notification. It applies equally to
> > > writability as to readability.
> >
> >   No true for Linux any more (this is the "thundering herd" problem).  Now,
> > only one thread/process is woken up---the one that actually received the
> > packet.
>
> The thundering herd problem still exists except under very narrow
> conditions. In particular it's only fixed where at least one thread is
> already blocking on a read or accept call (i.e. not polling). Only then can
> the kernel atomically clear the ready state and dequeue the data.

  Okay, so it avoid the thundering herd problem if you call accept() or
read() directly on a socket, but not through select()/poll()/epoll().  Okay,
got that.

> > > Another poor assumption people make is thinking they don't need to set UDP
> > > sockets to non-blocking when writing, or even bother with polling for write
> > > readiness. Because UDP is lossy they assume that the kernel will simply drop
> > > any packet that won't fit in the output queue, immediately returning control
> > > to the writing thread. Not true, at least not on Linux. OTOH, Linux as a
> > > default socket buffer size of 65536 which means, again, you only trigger
> > > this bug under very heavy UDP load.
> >
> >   I'm not sure what distribution you are using, but I'm still running a
> > 2.6.9 kernel (32b) at home, and it reports a default UDP buffer size of
> > 110592; at work I'm running Linux 3.2.0 (64b) and it reports a default UDP
> > buffer size of 229376.
> >
>
> Indeed. So for you it's 2x or 4x what I reported. On the most recent server
> I helped somebody fix this problem (a Java cluster doing billions of UDP
> requests daily), it was 64k, but perhaps that was something they set
> themselves.
>
> But you're only proving my point. The point was that the huge buffers common
> on Linux mask the problem except under heavy load. It's not until you begin
> to saturate the network and fill the buffer that your threads begin to stall
> when writing a UDP packet unless you set O_NONBLOCK.

  Hmm ... (rushes off to fix code ... )

  -spc ("Networking is hard!  Let's do math!")