The event namespace offers a generic mechanism for event handling. This is primarily used whenever you need to watch for multiple sources of events (e.g. multiple sockets) and cannot afford to block waiting on any single one to become ready.
The event handling mechanism in this module was designed and implemented by Mike Pall and Diego Nehab for Luasocket as a better alternative to the abominable socket.select() API.
Second draft. Any and all comments welcome. The API is still in flux and may change before the final version is released. —Mike
The main features of this API are:
To obtain the event namespace, run:
-- Loads the event module and everything it requires.
local event = require("event")
Event dispatching is sort of the opposite of using native threads and blocking I/O in a multitasking OS. With this paradigm you have to create one thread for each parallel flow of control:
With event dispatching you can use a single native thread to process multiple event sources. Events are handled synchronously one-after-another:
Well, as you can see, there ain't no such thing as a free lunch (TANSTAAFL). So let's stop comparing the relative merits of each approach now. Let's start to explain the concepts behind the event handling mechanism provided by this module.
First a bit of terminology:
An event container allows for adding, deleting and triggering events and provides an iterator to fetch triggered events.
An event has the following properties:
Here are the steps you have to do in order to use the event handling mechanism:
Here is a simple example:
-- Create a new event container with 1 associated object per event.
local ev = event.new(1)
-- Add some events to the container.
ev:add("t", 2.5, "One-shot timer expired")
local tid = ev:add("T", 5.0, "Periodic timer triggered")
local vid = ev:add("v", nil, "Virtual event triggered")
-- Process all triggered events in an infinite loop.
for id, etype, src, str in ev(true) do
-- Print the current time and the associated object.
print(socket.gettime(), str)
if id == tid then
ev:trigger(vid)
elseif id == vid then
ev:add("t", 1.0, "One second later ...")
end
end
Strings are used as associated objects for the sake of keeping this example simple. The body of the loop does nothing spectacular either. Better examples that show various event dispatching strategies will follow in the next sections.
The following table lists all supported event types:
| event type | event source | trigger condition | POSIX | Windows |
|---|---|---|---|---|
| "v", "V" | Virtual event | has been triggered | X | X |
| "t", "T" | Timer | has expired | X | X |
| "r", "R" | POSIX file | is readable | X | |
| "w", "W" | POSIX file | is writable | X | |
| "x", "X" | POSIX file | has an exception pending | X | |
| "s", "S" | POSIX signal | has been caught | X | |
| "h", "H" | Windows handle | state is signaled | X | |
| "m", "M" | Windows message | is in the input queue | X |
Event types are passed to the API as one-character strings. Lowercase letters indicate temporary events; these are deleted from the event container once the event has been triggered. Uppercase letters indicate persistent events; you have to explicitly delete them from the container.
There are other event sources imaginable. E.g. virtual events that work across threads or processes or virtual events with a counter. Extending the list with events triggered by other system resources is difficult, because you need a single call in the backend that checks for all events at once. Otherwise you are doomed to use polling. Ugh. More input is welcome. — Mike
A virtual event is a simple condition that can be either in the untriggered state or in the triggered state. The event source must be set to nil when adding a virtual event with ev:add. Virtual events can only be triggered with ev:trigger.
One-shot virtual events are deleted when they are returned from the iterator; persistent virtual events can be triggered multiple times and have to be deleted manually. A virtual event is not retriggerable until it is returned by the iterator. It will be returned only once by the iterator, no matter how many times it has been triggered in-between.
Note for POSIX: It is not safe to trigger events from inside signal handlers. Use the signal event source to trigger an event from a signal and add any required processing to a callback that is run when the signal event is synchronously returned from the iterator.
Note for Windows: It is safe to trigger a virtual event from an APC. Creating and using virtual events is cheap and should be preferred over using Windows event handles.
Virtual events are the best way to safely use overlapped I/O: When an overlapped I/O operation cannot complete immediately, create a virtual event and associate it with a callback that is to be called after the operation completes. You can pass the event id to the completion handler by re-using the hEvent parameter of the OVERLAPPED structure. Trigger the event when the completion handler is run inside an APC. It will be delivered synchronously through the iterator to the dispatcher which calls the associated callback.
A Timer is specified by a number that gives the time interval after which the timer expires. The time unit is one second; fractional values are allowed. The time interval is relative to the time the event was added. Temporary timers trigger an event only once; Persistent timers periodically trigger the event until you delete them from the event container.
Note: Timers never expire before the requested time, but may expire shortly after the requested time depending on system timer resolution and event queueing delays. In general you cannot rely on 100% accurate timing in a non-realtime operating system anyway.
A file is identified by a file descriptor. This can be a disk file, a pipe, a socket or any other file resource your OS provides. An associated event will be triggered when a file is readable, writable or has an exception pending, depending on the event type.
A file descriptor obtained from an external Lua module can be passed in as a number or as a LIGHTUSERDATA object.
Note: You have to make sure that the container only holds valid file descriptors at all times. You have to delete all untriggered events and all persistent events before closing the file descriptor.
Note: The results of adding the same file descriptor with the same event type to more than one event container are undefined.
A signal is identified by its signal number or its name. It is triggered when a signal is caught. Signal numbers are system specific. Signal names (like "INT" or "HUP") are more portable, but your OS may not provide certain signals and conversely the internal lookup list may not contain all names that your OS provides. You can get more details about the signals supported by your OS with 'man 7 signal' and 'kill -l'.
Note: A signal can only be added to one event container at a time.
All waitable Windows handles can be used as event sources. Please refer to the documentation of the Windows kernel function 'MsgWaitForMultipleObjectsEx' for the list of supported handle types. A handle obtained from an external Lua module is passed in as a LIGHTUSERDATA object.
Note: The results of adding the same handle to more than one event container are undefined.
Please give me some input about the most useful way to specify the selector for this event source and how it should map to dwWakeMask for MsgWaitForMultipleObjectsEx. Obviously allowing only QS_ALLINPUT is trivial (if all you want is to call some message queue handler). Allowing only single-bit selectors and mapping each of them to a single event is simple, too. But the semantics get pretty complicated if multi-bit selectors (such as QS_INPUT) need to be mapped to a single event. So, what are the requirements for this event type in your apps? — Mike
ev = event.new([numobj])
Creates and initializes a new event container.
The numobj parameter gives the maximum number of objects that can be associated with each event. The default is zero.
The function returns the newly created event container. See the section on Event container methods below.
Associated objects can be used to pass context information along with events. They are passed in when the event is added to the container and will be returned by the iterator when the event has been triggered. Associated objects are typically used by a dispatcher to tie an event to a callback, a state or a thread.
The event container needs to create numobj tables to store the associated object. Adding, deleting or iterating over events incurs additional overhead depending on the number of associated objects.
Note: Usually only a single event container is needed in a single process. However different event containers are independent of each other and can be used e.g. in different native threads in a shared Lua universe. It is not safe to access the same event container from different native threads without proper locking.
Note: The effects of adding the same system event source to different event containers are unpredictable. In general the set of event sources should not overlap between any two event containers in a single process.
Example:
-- Create a new event container with 1 associated object per event. local ev = event.new(1)
id = ev:add(type [, source [, obj*]])
Adds an event to the event container with the given event type, event source and associated objects. Depending on the event type a previous event with the same type and/or the same source may be replaced.
For details about the valid event types and sourcesl see the sections on Event types and Event sources.
Returns the event id for the newly created event as a LIGHTUSERDATA object.
The following examples show different ways to use associated objects. Each one of them needs an appropriate dispatcher that knows what to do with the associated object(s) returned from the iterator. I.e. call it or do some other processing.
-- Create a one-shot timer with a callback function.
ev:add("t", 10.0, function() print("Timer expired!") end)
-- Create a persistent timer with a table that holds a complex object.
local obj = { notify = "acceptor", rehash = true }
obj.id = ev:add("T", 60.0, obj)
-- Typical event mode socket reader.
local function reader(ev, obj)
local s, err, part, id = obj.sock:receive()
while s do
obj:receiver(obj.part .. s)
obj.part = ""
s, err, part, id = obj.sock:receive()
end
if err == "event" then
if part then obj.part = obj.part .. part end
ev:set(id, reader, obj) -- Set associated objects for the created event.
return
end
obj:onerror(err) -- Finalize the object and pass the error.
sock:close()
end
-- (Some initialization omitted for brevity)
obj.part = "" -- Initialize partial receive buffer.
obj.sock:seteventmode(ev) -- Put the socket into event mode.
ev:once(reader, obj) -- Run the reader once to setup the transfer.
POSIX only:
-- Create an event for file readability. The file descriptor must be
-- a number or a LIGHTUSERDATA object obtained from elsewhere.
ev:add("r", fd, reader)
-- Create events to catch SIGINT (one-shot) and SIGHUP (persistent)
ev:add("s", "INT", function() print("Goodbye!") os.exit() end)
ev:add("S", "HUP", function() db.reload(true) end)
Windows only:
-- Create an event that is triggered when a semaphore is signaled. The
-- semaphore handle must be a lightuserdata object obtained from elsewhere.
ev:add("h", sem, function() shared_obj:handler() end)
-- Create an event that is triggered when a WM_*KEY* message is in the queue.
ev:add("m", "KEY", function() gui.poll(0.001) end)
id = ev:set(id [, obj*])
Sets or overrides the associated objects for an existing event.
The event id specifies the event that is to be associated with the given objects.
Returns the modified event id as a convenience. Returns nil if the event id is invalid or has been deleted.
Example:
-- Add a persistent timer with a callback function.
local tid = ev:add("T", 2.5, function() print("Timer expired!") end)
-- Later, somewhere else, maybe in a callback: Modify the callback.
ev:set(tid, function() print("Oh no, the timer expired again!") end)
id = ev:trigger(id)
Trigger a virtual event. The event will be delivered synchronously by the iterator.
The event id specifies the event to trigger.
Returns the id of the triggered event as a convenience. Returns nil if the event id is invalid, has been deleted or does not refer to a virtual event.
The C API only requires the specification of the event id. Neither the event container object nor the Lua state is required because event ids are unique across all event containers of a single process.
Example:
-- Add a persistent virtual event.
local vid = ev:add("V", nil, function() print("Triggered!") end)
-- Later, somewhere else, maybe in a callback: Trigger it (once).
ev:trigger(vid) -- Event will be returned in dispatcher loop.
id = ev:once([obj*])
Generate an immediate one-shot event. This is implemented by adding a virtual event and triggering it. The event will be delivered synchronously by the iterator.
The event is added with the specified associated objects.
Returns the id of the added event.
Example:
-- Arrange for calling two functions from the dispatcher loop.
ev:once(function() print("Hello world!") end)
ev:once(function() print("The current time is:", os.date()) end)
id = ev:del(id)
Deletes an event from the event container.
The event to be deleted is specified by the event id.
Returns the id of the deleted event as a convenience. Returns nil if the event id is invalid or has already been deleted.
Example:
-- Add a persistent timer that prints a counter every second.
local cnt = 0
local tid = ev:add("T", 1.0, function() cnt = cnt + 1; print(cnt) end)
-- Later, somewhere else, maybe in a callback: Delete the timer to stop it.
ev:del("t", tid)
ev:stop()
Stops any running iterator.
This method can be used anywhere inside the iteration loop, e.g. in callback functions. It forces any running iterator to return nil on the next invocation which terminates the loop. Note that any pending triggered events are not deleted. Creating a new iterator and restarting the loop will just continue returning them.
ev:clear()
Deletes all events from an event container.
This method can be used from anywhere, e.g. inside the iteration loop or in callback functions. Pending triggered events are deleted, too. A surrounding iteration loop will abort since the iterator returns nil when the container is empty.
The container can be reused and new events can be added immediately after calling ev:clear(). A surrounding iteration loop will still abort if non-blocking behaviour is selected since a new event collection cycle has to be started.
iterator, ev, nil = ev([block])
id, type, source [, obj*] = iterator(ev, id)
The event container object can be called and returns an iterator. Calling the iterator returns triggered events from the container.
The block parameter specifies the blocking behaviour of the iterator. true = block. false = never block. nil (default) = block only if the container holds at least one timer event.
Three values are returned by calling the event container object: The iterator function, the event container object and nil. The iterator function returns the event id, the event type, the event source and any associated objects.
Each call to the iterator function returns a single event that has been triggered. The iterator returns nil if a) the container is empty or b) non-blocking behaviour was selected and no triggered events from the current event collection cycle are remaining.
The iterator is destructive because it removes triggered events and may also delete events from the container as a result. Do not use multiple iterators over the same container in parallel. Aborting the loop and creating a new iterator is allowed though. It is safe to call other methods even while using an iterator over the same event container.
Iterators are typically used in for loops and that takes care of managing the iterator object. Try not to be confused by the above description, just cut'n'paste one of the examples:
-- Careful: Using 'type' as a variable name will override the global -- function of the same name. Using 'etype' is a better choice. -- Process all triggered events in an infinite loop. for id, etype, src, obj in ev(true) do ... end -- One-shot loop through all events that have been triggered (non-blocking). for id, etype, src, obj1, obj2 in ev(false) do ... end -- Do not wrap this up into another loop unless the other loop blocks -- for some time. Otherwise you get polling behaviour.
The event handling mechanism implemented by this module is passive. This means that it does not invoke any actions on its own (like invoking a callback function). I.e. it is not an event dispatcher. This is left up to your application or up to other higher-level modules.
Such a coroutine-based higher-level module is in the works and will be announced whenever it is ready. It is not yet clear whether this will become a part of Luasocket or will be released in a separate package. — Mike
While this may seem a little bit inconvenient at first, here is a good reason for this design decision: Higher Flexibility! There is just no single dispatching strategy that fits every application.
The advantage of a passive event handling mechanism is that it can be easily adapted to any kind of event dispatching strategy. The other way round is generally a lot harder. If you have ever tried to add (say) a thread dispatcher on top of a callback-oriented event loop, you'd know what I'm talking about.
Many different event dispatching strategies have been invented. Here is a small selection:
One-Big-Switch: All control-flow relevant processing is done from a single main loop in a single function. Context is kept in local variables.
That's the way some network servers have been implemented. I guess most event dispatchers started this way and have moved on to the callback model later. Maybe ok for your pet project. But please use a strategy that provides a better abstraction for anything bigger.
Callbacks: A callback function is invoked whenever an event triggers. The intra-call context has to be kept in state variables that have to be passed around.
This strategy is commonly used for GUIs and quite a few network servers. Good for simple tasks. Breaking the control flow up into many small snippets and passing the context quickly becomes annoying for anything more complicated.
State machines: A triggered event is interpreted as a state machine transition. Depending on the current state and the transition some processing is done and then a new current state is set. The intra-state context has to be kept in the state machine or in other variables that have to be passed around.
Most useful for processing complicated and layered protocols. Designing and debugging a state machine is not for the faint-hearted.
Continuations: This mechanism depends on implementation language support for closures that can be returned as first class objects and/or proper tail calls (Lua has both). A triggered event invokes the continuation that has been passed to the dispatcher from a previous invocation. The closures keep the context between continuations.
Rather uncommon and there are some pitfalls (like loops). Please report if you have survived.
Coroutine dispatcher: A specific coroutine is resumed whenever an event triggers. This coroutine yields back to the dispatcher when it needs to wait for an event. The context is kept in the stack of the coroutine.
Allows you to keep the linear flow of control. Depends on good support for coroutines and the ability to yield from anywhere in the call stack. Lua qualifies (with the exception of pcall). Most other languages don't and this is the only reason why it's uncommon. It's extremly convenient and marries the advantages of the event based model and the native thread model. Can you tell I like it? :-)
TBD