Re: Single-threading servers and select()

[skaller <skaller@...>]

On Sun, 2005-01-23 at 15:31, skaller wrote:
> On Sun, 2005-01-23 at 11:52, duck wrote:
> > Recently, it was suggested that select()-driven TCP servers could never 
> > scale to many 1000s of connections because of the linear cost of searching 
> > the select array each time a non-zero number of connections had some data
> > handy.
> > 
> > In the real world, is this really an intractable problem? 
> 
> Yes and no :)

BTW: I would like to add something here. There is
a web-server based problem which is a higher level
generalisation of the connection problem:
maintaining many simultaneous *sessions*.

Even when HTTP connections are transient, most
e-commerce systems require a logical connection
(called a session) to be persistent.

The issue here is not implementation details or
performance, but programmability -- you would like
to write your interaction with the client
as if you were talking to them .. not as if you
got locked up in a cupboard between each sentence,
and had to emerge each time and reconstruct the
conversation.. :)

A very high level system was designed to support
making this transparent. By high level .. I mean
it required some heavy duty category theory which
replaced Haskell monads with a new more powerful
concept (called Arrows -- i.e. this is bleeding
edge academic research ..)

So this problem of 'scalability' is not simple,
and isn't just a 'select vs poll vs kqueue' problem:
the problem is much more fundamental than that.

I.e. Lua, Felix, Cappriccio, Io, and most other
such systems that recognize the need for coroutines
all use different models because we simply don't
know how to use them due to a lack of theory.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net