[ANN] cqueues 20140729

[William Ahern <william@...>]

I recently tagged a new release of cqueues, a non-blocking event framework
for modern Unix systems. I wanted to wait until I finished adding O(1)
timeouts (worst case, any operation), making it the only open source event
framework in any language to have O(1) timeouts. But I haven't yet had the
time to finish the integration.

Notable recent additions include:

* Promise/Future module patterned after C++11. Not strictly necessary
  when you have coroutines, but eases writing things like caches. Built
  around the Condition variable module added in the last big release.

* DNS resolver pools. The underling non-blocking resolver library uses one
  socket per query for security and to simplify descriptor management. For
  DNS-heavy applications it was suggested to use a resolver pool in order
  not to not waste descriptors. Now the pool interface comes built-in. The
  resolver pool module has been running in a high-traffic (millions of
  queries per day per instance) DNS-heavy cloud service for several weeks.

* All buffered I/O routines now support timeouts. Note that this slightly
  changed the behavior of :starttls.

* HTTP multipart MIME body parsing is now much easier and more performant
  thanks to a specialized socket read format which takes a MIME boundary and
  returns chunks until the boundary is found.

* Text-mode block reads now should behave just like C's stdio. That is, if
  1024 bytes are requested, then 1024 bytes are returned, which may have
  necessitated reading more than 1024 bytes from the socket given any EOL
  translations or to ensure a trailing carriage return is not followed by a
  linefeed. Previously all block reads were performed as-if in binary mode,
  without any EOL translation.

* Fast metatable checking in the sockets module using upvalues.

* Functions can now be passed through thread.start. dladdr+dlopen is used to
  pin C functions in memory to make things thread-safe. Otherwise a function
  pointer could become invalid if the Lua VM it was sent from unloaded the
  module.

* Socket module now preserves errors across invocations to prevent buggy
  applications from silently dropping data. :clearerr should be used when a
  recoverable error is encountered and dealt with.

* Socket module now throws an error when it detects too many consecutive
  uncleared errors. This is a back-stop to help catch bugs exposed by the
  previous feature of preserving errors.

Homepage: http://25thandclement.com/~william/projects/cqueues.html
Documentation: http://25thandclement.com/~william/projects/cqueues.pdf

About cqueues:

cqueues is a high-performance, coroutine based event loop which wraps epoll, 
kqueue, and Solaris ports event descriptors, as well as their cousin 
interfaces such as signalfd and inotify. 

* Composable: Each event loop is itself pollable, and can be easily tied 
  into existing event loop systems in the larger application, whether in 
  Lua, C, or some other language. loop:pollfd() returns the underlying 
  epoll, kqueue, or ports descriptor, and will signal ready when a child 
  coroutine is ready to resume. 

  Additionally, the modules maintain no global state in the user process, 
  nor will they ever block the process. No callbacks are used, so C code 
  within or outside the event loop does not need to worry about reentrancy. 

  You can synthesize pollable objects by providing the methods :pollfd, 
  :events, and :timeout. 

* High-performance: Because the loop controller pins every polled object 
  into memory for at least one full scheduling cycle, descriptor management 
  is optimal, without recourse to one-shot polling or complex reference 
  counting. 

* No Dependencies: The only dependencies needed are already installed on 
  your server--pthreads, OpenSSL, and of course Lua! 

* Comprehensive: The package provides modules for sockets, signals, native 
  file change notification, POSIX thread management, DNS resolution, and 
  light-weight Lua thread synchronization. 

  + cqueues.socket: Full-, line-, and non-buffering sockets with CRLF/LF 
    translation, automagic DNS resolution, SSL/TLS capabilities, and Unix 
    file descriptor passing. 

  + cqueues.signal: Employs Linux signalfd and BSD EVFILT_SIGNAL. 
    On Solaris synthesizes non-blocking signal polling with sigtimedwait. 

  + cqueues.notify: Employs Linux inotify, BSD EVFILT_VNODE, and Solaris 
    PORT_SOURCE_FILE for efficient file change notifications. 

  + cqueues.thread: Simple POSIX thread management, with pollable 
    thread handles and automatic socket channel creation for easy 
    inter-thread communication. As the sockets module supports descriptor 
    passing, it's trivial to transfer open files or sockets between POSIX 
    threads. 

  + cqueues.dns: Full DNS resolver (stub and recursive) with ability 
    to configure itself from common /etc configuration files on BSD and SysV 
    systems. 

  + cqueues.condition: Light-weight, user-defined condition variables 
    for efficient synchronization of Lua threads. 

  + cqueues.promise: Promise/Future pattern module.

* Maintained: cqueues is used to power multiple commercial services and 
  products with significant traffic. Bugs are fixed quickly, and new 
  features added on a steady basis. This and the non-open source predecessor 
  project have been in continuous use and development for nearly 4 years.