Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

[William Ahern <william@...>]

On Mon, Aug 25, 2014 at 04:54:53PM -0700, Coroutines wrote:
> On Mon, Aug 25, 2014 at 4:38 PM, Sean Conner <sean@conman.org> wrote:
<snip>
> >   read() is fine for TCP and for "connected" UDP sockets [1], but for
> > unconnected TCP (or even packets for other IP protocols like OSPF) you can't
> > use read().  That's one reason I rejected using read().
> 
> Secret:  You can associate the remote peer (sockaddr) with a UDP
> socket by calling connect() -- after that I believe you can call
> read() like normal.  The man page says you can only connect() once for
> TCP sockets, but you can call connect() multiple times for UDP
> sockets.  This results in the kernel associating the remote peer with
> that socket in the networking stack -- outwardly it doesn't seem to
> have any effect.  People think because you only have an fd (an int)
> that nothing happened -- buwhahaha bsd socket juju.  I know for sure
> is how you can use recv() instead of recvfrom() on a UDP socket.
> 

This is broken on Linux, although I'm not sure when this problem was
introduced. I brought it up on the netdev mailing list last year and nobody
seemed to care.

If you try to reassociate a socket from a loopback address to a non-loopback
address then connect will fail. The problem resides in the kernel handling
of loopback device binding. I couldn't figure it out but somebody else on
comp.unix.programmer tracked it down. It actually only happens if the
_first_ bind was to a loopback address.

I originally stumbled upon this issue with my DNS resolver. In the resolver
I would simply reassociate the UDP socket using connect. However, if
somebody had an /etc/resolv.conf like

	nameserver 127.0.0.1
	nameserver 8.8.8.8

then it would return an error on the rare occassion it failed over to
8.8.8.8. You can see the behavior using the example code I've pasted at the
end of the message.

$ ./udp 127.0.0.1 8.8.8.8
connect(127.0.0.1): OK
connect(8.8.8.8): Invalid argument

$ ./udp 8.8.8.8 127.0.0.1
connect(8.8.8.8): OK
connect(127.0.0.1): OK

I've only seen this behavior on Linux. On every other systems I've tried--OS
X, *BSD, Solaris, AIX--you get the expected behavior.

The "fix" is to first unbind the socket by connecting it to AF_UNSPEC.
However, keep in mind that some systems (e.g. OS X) will return an error
when calling connect with an AF_UNSPEC address faily. So you either have to
do this using platform-specific logic, or ignore any failure when
"unbinding" the socket (or both, because I'm not sure which kernel version
or patch introduced this behavior).

Here's sample test code:

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <err.h>

int main(int argc, char *argv[]) {
  int i, fd;
  struct sockaddr_in sin;

  if (-1 == (fd = socket(AF_INET, SOCK_DGRAM, PF_UNSPEC)))
    err(1, "socket");

  for (i = 1; i < argc; i++) {
    sin.sin_family = AF_INET;
    sin.sin_port = htons(53);
    sin.sin_addr.s_addr = inet_addr(argv[i]);

    if (0 != connect(fd, (void *)&sin, sizeof sin))
      printf("connect(%s): %s\n", inet_ntoa(sin.sin_addr), strerror(errno));
    else
      printf("connect(%s): OK\n", inet_ntoa(sin.sin_addr));
  }

  return 0;
}