Re: [ANN] Prosody 0.8.0 released

[Matthew Wild <mwild1@...>]

Ed sent this to me off-list, but suggested that he shouldn't have as
my response was worth a wider audience.

In fact I am also interested to know if anyone has experience of the
issues below, or the various libraries. I did try LuaLanes a couple of
years back, but only got it to deadlock. Now that it is somewhat more
maintained now I should look into it again when I get the chance.

On 8 April 2011 17:33, Ed <spied@yandex.ru> wrote:
> On 04/07/11 18:56, Matthew Wild wrote:
>>
>> Hi,
>>
>> We are very pleased to announce the release of Prosody 0.8.0! Many thanks
>> to the many people who made this release possible.
>
> sorry for dump question ;)
>
> what multitasking model prosody use?
> it seems to not use fork or threads.
>

It does not currently fork or use threads, no. This is primarily
because we are not certain of the best way to do it. The "best" in
this case being the way that gives us the best performance for the
least code complexity (ideally little to none of the existing code
would be changed).

Forking isn't a good option, as unlike in a HTTP server in XMPP all
the connections may need to frequently exchange data with each other.
We also couldn't really scale by forking for each client (we should be
able to handle hundreds of thousands of concurrent connections), so
there would also be some code complexity in pooling and balancing load
between a (presumably fixed) number of processes.

Threading is a more viable alternative, but again there are many
different ways to do it, some options we have considered:

* One thread handling connections, and multiple worker threads for
processing messages from a queue (producer/consumer style):
 This approach may not bring much performance as it may end up with a
number of global locks around things like storage and network. Most
code would be untouched, except for where locks are required.

* Thread pool for specific blocking tasks, such as storage (few
storage truly async storage APIs are available)
  This is currently our favoured approach. We just scale out the few
parts we know can be bottlenecks, the majority of work still happens
in one main thread, meaning no locking or other threading issues can
appear so easily.

* What I call the "magic" LuaProc/erlang style, where we can just put
each session in a coroutine, and let the scheduler deal with the
multi-CPU scaling.
 This is attractive as it requires very little work on the threading,
we can just focus on using coroutines and message passing. It should
also be possible to easily scale down to 1 thread.

The main reason we haven't jumped on any one of these is that most of
them would be a lot of work to implement, and we have no hard data on
which of them are going to bring us the most benefit without messing
up our clean single-threaded (and already fast) code. Implementing
threading in the wrong way could actually hurt performance, because of
communication between threads slowing the system down.

And of course another significant reason is that CPU is rarely the
bottleneck for an XMPP server, it's typically RAM (when you have tens
of of thousands of clients with various state, you do need a decent
server RAM-wise). So scaling across CPUs has yet to be a priority for
us.

More of a priority is scaling across multiple machines - ie. having
multiple Prosody instances serving the same domain (for load-balancing
and reliability). This would also allow multiple Prosodies to run on
one machine and share the CPUs.

> prosody use concurrent multistaking based on select (or something similar)?
> it is not like copas too - coroutines are rarely used.
>

By default Prosody uses non-blocking connections and select(), and if
you install luaevent then we also have a libevent backend as well
which allows us to use epoll/kqueue/etc. for greater efficiency.

We don't use a coroutine per connection like copas does, we use
callbacks instead. Some rough benchmarks when we first started hinted
that we could save quite a bit on resources by not using coroutines.

On the other hand Prosody originally started with copas as it was nice
and simple, but I switched to `server.lua` from the luadch project
when I failed to get copas working with non-blocking encrypted
connections.

We've since adapted server.lua extensively to provide various
connection-related APIs, and abstract things like efficiently linking
sockets together (like stream.pump if you're familiar with node.js).

Hope this helps,
Matthew