Re: [ANN] lsocket - Another library for sockets in lua

[William Ahern <william@...>]

On Wed, Mar 27, 2013 at 09:08:41PM -0400, Sean Conner wrote:
<snip>
>   True story.  At work, two developers (on two different projects) were both
> trying to use an existing DNS resolving library (c-ares).  Both were having
> major difficulty in getting it to work and it all came down to the network
> code in the library.  Not only was it trying to do too much, but it didn't
> mesh well with the existing network stack in the application.  

That was my story, too. Callbacks are just not composable. The operating
system guys figured this out in the 70s. GUI writers are still playing catch
up, with Lua developers leading the way!

It doesn't help, though, that Microsoft still pushes hardware-asynchronous
callbacks. There's a reason why they made SSL a kernel service; it's a
nightmare to stitch it into the execution path in userland, w/ umpteen
levels of callbacks. Same reason why integrating ADNS or c-ares into complex
network stacks is so difficult.

>   I'm sure that c-ares would be fine for an application that wasn't so
> network heavy as both projects (both in telephony [2]).  But with SPCDNS,
> since it doesn't really handle the network portion of DNS, was much easier
> to integrate into the projects (literally within a couple of days) since
> they could integrate the DNS calls with the rest of the network stack in
> place.

Copy that. I write _all_ of my libraries to be non-blocking, restartable
state machines. DNS, HTTP, RTSP, SMTP, even MIME. I write them as push-pull
parsers. Basically, a method to _push_ data into the machine, and then a
method to _pull_ out the enqueued tokens/events. This makes them
immeasurably easier to integrate into applications. And you can easily layer
all kinds of I/O patterns atop this, including tiny auxiliary wrappers which
handle socket I/O.**

When I began writing dns.c, this is how I wanted to do it--no instrinsic
socket support whatsoever. Aside from providing all the packet and record
code as a resuseable API, I wanted to be able instantiate a resolver
instance which consumed and emitted operations to be performed by the
caller. But this simply proved too unwieldly. The socket operations were
simply too complex (complex timeouts, multiple exchanges with multiple
peers), and I figured I might as well put it all in one place rather than
recreate it in every application.

But I can totally appreciate electing to go in the other direction, slimming
the library down rather than beefing it up.

>   What I've noticed (having looked at most of the existing DNS resolving
> libraries) is that they tend to skimp on decoding DNS records (leaving most
> of that up to the application) and go to extreme lengths in handling the
> network side of the protocol.  SPCDNS took the opposite approach---handing
> the DNS decoding [3] and leaving the network side to the application.

I can't dispute your analysis. We all prefer different tradeoffs.
 
<snip>
> [3]	It handles 32 DNS record types---about 26 more types than other DNS
> 	resolving libraries.

I can't find any code which handles NAPTR pattern matching in SPCDNS. Do you
just use the POSIX regex library in your applications? (Presuming you
actually use NAPTR records.) I was always afraid to execute regular
expressions from untrusted third-parties, _especially_ with builtin POSIX
regex engines, which are mostly pretty horrible.


** I cringe most of the time when I hear people discussing code abstraction
and modularity. Beyond basic encapsulation, people abstract the wrong stuff.
They usually spend no time whatsoever thinking about how to support
different concurrency, parallelism, or code flow patterns, and instead
obsess over useless OOP abstractions. I can _easily_ fix poor object or
encapsulation abstractions by hacking the code. It's usually impossible to
address a poor approach to execution flow or data management.

So, I totally understand your approach to SPCDNS.