Re: scalability

[Matthew Wild <mwild1@...>]

On Tue, Apr 28, 2009 at 11:28 PM, Ivan-Assen Ivanov <ivanassen@gmail.com> wrote:
>> Is lua a suitable language for this target domain?
>
> My gut feeling is that nowadays it's quite possible to achieve the
> goal of the C10K paper, that is, to serve 10k clients from a single
> machine - in Lua. Don't let the small problem of having to serve
> thousands of clients drive you into using an inferior language!
>

I couldn't have said it better. I've been half-following this thread
with interest, but hadn't yet the time to form a reply.

I develop Prosody, an XMPP server written in Lua. We're finding that
we compare quite well against other servers (including passing one
written in erlang). ...right up to the point where we get hit by the
limitations of select(). I'm quite confident that once we find a
solution for this (LuaSocket's infamous 3.0 being a likely
possibility) and also for scaling over multiple CPUs (and possibly
machines) then we'll be doing marvellously.

I do get a lot of comments on the choice of language for Prosody. I
answer all by pointing out Lua's speed and small footprint. Why aren't
these advantageous in server development? Also... people have written
servers in Ruby and Python, I know for sure that Lua is *definitely*
*not* worse than these languages :)

I've also seen erlang servers performing *really* badly. Allocating
gigabytes of RAM (about 3GB) every few seconds, and then every so
often the gc (non-incremental) kicking in and reducing the RAM usage
by over 8GB in a sweep.

Language choice doesn't affect performance nearly so much as people
imagine. It's more about the application implementation, and a lot of
people still don't get this right, no matter what language they use.

Matthew.