Re: Workshop slidedecks

[Daurnimator <quae@...>]

On 17 October 2015 at 12:20, Rob Kendrick <rjek@rjek.com> wrote:
>
> Hi,
>
> Does anybody know if the slides from each of the talks will be made
> available?  Some of my notes are not as detailed as I would like.
>
> B.
>

I've attached the source for my 2 talks.
compile to pdf with:

pandoc -i -t beamer -V theme:Warsaw a-modern-lua-library.md -o
a-modern-lua-library.pdf
pandoc -i -t beamer -V theme:Warsaw cqueues.md -o cqueues.pdf

% A Modern Lua Library
% daurnimator
% Friday, October 16, 2015

# Good projects:

  - come in library form
	  - are not their own executable
  - are composable i.e. work well with any other library
  - are well tested
  - are well documented (oh crap...)


# Must be composable

  - No globals
	  - Use luacheck
  - No changing global metatables (e.g. string or lightuserdata)
  - No "only one" of anything (e.g. main loop)
  - Can be tricky if you need more than one thing going on at a time
	  - see my next talk: cqueues


# Preferred tooling

(in 2015)

  - Version Control
	  - git
  - Hosting, Issue Tracking
	  - [github](http://github.com)
  - Tests
	  - [busted](http://olivinelabs.com/busted/)
  - Use a CI (Continuous Integration) system
	  - [travis](http://travis-ci.org)
	  - [appveyor](http://appveyor.com) (for windows)
  - Do coverage analysis
	  - [luacov](https://github.com/keplerproject/luacov)
	  - [coveralls.io](https://coveralls.io/)
  - Use [luacheck](https://github.com/mpeterv/luacheck)
  - Make a rockspec
	  - Upload it to [luarocks.org](https://luarocks.org/)


# Quick set up

Moteus has a great starting point: https://github.com/moteus/lua-travis-example/

  - Copy `.travis` directory into your project
  - Customise the `.travis.yml`
	  - add whatever dependencies you have
	  - run your tests in `script` section


# Getting your library into the hands of others

Tell people about what you've created!

  - Host on github unless you have a great reason not to; discoverability is good
  - Send an email to the lua list with `[ANN]` in the subject
  - Tweet about it
  - Write a blog post about it
  - Mention it at a conference ;)

# Case study: [fifo.lua](https://github.com/daurnimator/fifo.lua)

A simple, single file library that implements a FIFO of lua objects

  - busted
  - luacov
  - luarocks
  - git hooks
	  - `pre-commit`: run tests (quick; only for my currently installed lua version)
	  - `pre-push`: uploads scm rockspec
  - documentation is in markdown
	  - can be compiled to a man page with [pandoc](http://pandoc.org)
  - travis ci
  - coveralls


# Other more subjective notes

  - Write for the latest version of lua (5.3) and backport
	  - use [compat-5.3](https://github.com/keplerproject/lua-compat-5.3)
  - Prefer coroutines and iterators to callbacks

When binding a C library:

  - write the thinnest possible C bindings to *all* exported functions
	  - can usually get away with documentation for this layer as "read the C docs"
  - write lua on top that provides a more idiomatic function signatures
	  - this should have well written documentation and examples

% cqueues
% daurnimator
% Friday, October 16, 2015

# coroutine scheduler

  - At it's core, provides the abstraction of `poll`
  - Works like POSIX `poll(2)`; but by yielding coroutines
	  - Under the hood; uses `epoll` on linux, `kqueue` on BSDs and OSX, `ports` on solaris
  - "cqueues interface": `pollfd`, `events`, `timeout`
  - Importantly: a cqueues scheduler has this interface *itself*
	  - i.e. you can have multiple event loops that run each other
	  - can expose from your library

In addition you have condition variables for synchronising coroutines


# cqueues interface

  - `pollfd`, `events`, `timeout` form the common low level element of integration on unix originated systems
  - If you look at linux; things have been moving in this direction for years
  	  - bsd sockets are classic example
	  - `signalfd` for signals
	  - `eventfd` for synchronisation
	  - `timerfd` for timers + the time
	  - `inotify`
	  - lots more!

Most C libraries expose this same 3 element interface:

  - libpq (official postgresql C client library)
  - curl's "multi" interface
  - libsystemd


# Composable

As a cqueues scheduler itself has the same cqueues interface,
to integrate with another mainloop, we just need a way to provide a file descriptor to wait on.

I have written example integrations with main loops such as:

  - [prosody's main loop](https://hg.prosody.im/trunk/file/tip/net/cqueues.lua) (both luasocket `select` based and luaevent)
  - [luv](https://github.com/luvit/luv/commit/6aadfa8c1abd6bf123330676118c6cbb2f92f2e6) (libuv)
  - [lgi](https://gist.github.com/daurnimator/11e356b98b7baefae10a) (GLib)

  - I have a [pull request open against ngx_lua](https://github.com/openresty/lua-nginx-module/pull/450) to add the missing interface
  - Talked to @kostja yesterday about exposing [the required interface](https://github.com/tarantool/tarantool/blob/48b98bb977516b90c82964f66d0fa31ceae92e17/src/lua/bsdsocket.cc#L398) there


# cqueues built in libs

cqueues includes high quality libraries that submit to the cqueues interface

  - `socket`
  - `dns`
  - `thread`
  - `signal`
  - `notify` (file changes)
  - `promise` (C++ style, not JS; is a simple wrapper around condition vars)


# My libraries using cqueues

If I write a library that need async operations; I now use cqueues

  - [cqueues-pgsql](https://github.com/daurnimator/cqueues-pgsql)
	  - wrapper around marc balmer's [luapgsql](https://github.com/mbalmer/luapgsql) using only async operations
  - [lredis](https://github.com/daurnimator/lredis) (redis client)
  - [lua-http](https://github.com/daurnimator/lua-http)


# Conclusion

Start using it!

Even if you don't depend on cqueues itself,
export the `pollfd`, `events`, `timeout` from your libraries

  - homepage: http://25thandclement.com/~william/projects/cqueues.html
  - github: https://github.com/wahern/cqueues