RE: [ANN] cqueues 20140729

["Christian A. Steiner" <christian.steiner@...>]

> making it the only open source event
> framework in any language to have O(1) timeouts.

could you elaborate a bit more on this? do you mean custom timeouts .. aka timers .. or just a timeout for an i/o operation?

________________________________________
From: lua-l-bounces@lists.lua.org [lua-l-bounces@lists.lua.org] On Behalf Of William Ahern [william@25thandClement.com]
Sent: Friday, August 01, 2014 6:18 PM
To: Lua mailing list
Subject: [ANN] cqueues 20140729

I recently tagged a new release of cqueues, a non-blocking event framework
for modern Unix systems. I wanted to wait until I finished adding O(1)
timeouts (worst case, any operation), making it the only open source event
framework in any language to have O(1) timeouts. But I haven't yet had the
time to finish the integration.

Notable recent additions include:

* Promise/Future module patterned after C++11. Not strictly necessary
  when you have coroutines, but eases writing things like caches. Built
  around the Condition variable module added in the last big release.

* DNS resolver pools. The underling non-blocking resolver library uses one
  socket per query for security and to simplify descriptor management. For
  DNS-heavy applications it was suggested to use a resolver pool in order
  not to not waste descriptors. Now the pool interface comes built-in. The
  resolver pool module has been running in a high-traffic (millions of
  queries per day per instance) DNS-heavy cloud service for several weeks.

* All buffered I/O routines now support timeouts. Note that this slightly
  changed the behavior of :starttls.

* HTTP multipart MIME body parsing is now much easier and more performant
  thanks to a specialized socket read format which takes a MIME boundary and
  returns chunks until the boundary is found.

* Text-mode block reads now should behave just like C's stdio. That is, if
  1024 bytes are requested, then 1024 bytes are returned, which may have
  necessitated reading more than 1024 bytes from the socket given any EOL
  translations or to ensure a trailing carriage return is not followed by a
  linefeed. Previously all block reads were performed as-if in binary mode,
  without any EOL translation.

* Fast metatable checking in the sockets module using upvalues.

* Functions can now be passed through thread.start. dladdr+dlopen is used to
  pin C functions in memory to make things thread-safe. Otherwise a function
  pointer could become invalid if the Lua VM it was sent from unloaded the
  module.

* Socket module now preserves errors across invocations to prevent buggy
  applications from silently dropping data. :clearerr should be used when a
  recoverable error is encountered and dealt with.

* Socket module now throws an error when it detects too many consecutive
  uncleared errors. This is a back-stop to help catch bugs exposed by the
  previous feature of preserving errors.

Homepage: http://25thandclement.com/~william/projects/cqueues.html
Documentation: http://25thandclement.com/~william/projects/cqueues.pdf

About cqueues:

cqueues is a high-performance, coroutine based event loop which wraps epoll,
kqueue, and Solaris ports event descriptors, as well as their cousin
interfaces such as signalfd and inotify.

* Composable: Each event loop is itself pollable, and can be easily tied
  into existing event loop systems in the larger application, whether in
  Lua, C, or some other language. loop:pollfd() returns the underlying
  epoll, kqueue, or ports descriptor, and will signal ready when a child
  coroutine is ready to resume.

  Additionally, the modules maintain no global state in the user process,
  nor will they ever block the process. No callbacks are used, so C code
  within or outside the event loop does not need to worry about reentrancy.

  You can synthesize pollable objects by providing the methods :pollfd,
  :events, and :timeout.

* High-performance: Because the loop controller pins every polled object
  into memory for at least one full scheduling cycle, descriptor management
  is optimal, without recourse to one-shot polling or complex reference
  counting.

* No Dependencies: The only dependencies needed are already installed on
  your server--pthreads, OpenSSL, and of course Lua!

* Comprehensive: The package provides modules for sockets, signals, native
  file change notification, POSIX thread management, DNS resolution, and
  light-weight Lua thread synchronization.

  + cqueues.socket: Full-, line-, and non-buffering sockets with CRLF/LF
    translation, automagic DNS resolution, SSL/TLS capabilities, and Unix
    file descriptor passing.

  + cqueues.signal: Employs Linux signalfd and BSD EVFILT_SIGNAL.
    On Solaris synthesizes non-blocking signal polling with sigtimedwait.

  + cqueues.notify: Employs Linux inotify, BSD EVFILT_VNODE, and Solaris
    PORT_SOURCE_FILE for efficient file change notifications.

  + cqueues.thread: Simple POSIX thread management, with pollable
    thread handles and automatic socket channel creation for easy
    inter-thread communication. As the sockets module supports descriptor
    passing, it's trivial to transfer open files or sockets between POSIX
    threads.

  + cqueues.dns: Full DNS resolver (stub and recursive) with ability
    to configure itself from common /etc configuration files on BSD and SysV
    systems.

  + cqueues.condition: Light-weight, user-defined condition variables
    for efficient synchronization of Lua threads.

  + cqueues.promise: Promise/Future pattern module.

* Maintained: cqueues is used to power multiple commercial services and
  products with significant traffic. Bugs are fixed quickly, and new
  features added on a steady basis. This and the non-open source predecessor
  project have been in continuous use and development for nearly 4 years.