Re: [ANNOUNCE]: LuaSocket 2.0.3 RC2

[Jerome Vuarand <jerome.vuarand@...>]

Just a few comments:

2009/11/4 Nicolas <nicolas@net-core.org>:
>> > So you could accept connections in a given Lua state and process it in a
>> > different one. Be it by means of using Lanes, LuaProc, or whatever suits
>> > you.
>>
>> The above doesn't answer the OP's question of how
>>
>> fd = ssock:acceptfd()
>>
>> is different from
>>
>> fd = ssock:accept():getfd()
>>
>> It might be that the problem is that the accepted userdata will close
>> the underlying fd in it's __gc().
>>
>> If true, acceptfd() might be better replaced with a method of
>> relinquishing ownership.
>
> There is currently no way that I know of to make luasocket work
> corrently for this behavior.
> getfd and setfd are not good enough for that (as Diego says they
> were made for debugging mainly).
>
> When you do ssock:accept():getfd() you risk getting the fd closed
> by the GC as you said, but that is not all.
> Then you must use that fd somehow, so you try to do:
> s = socket.tcp()
> s:setfd(fd)
> data = s:recieve()
>
> But then receive complains that s is of type tcp{master} instead
> of tcp{client}. Also, socket.tcp() does actually create a fd when
> called, not when :connect() or :bind() are called.
> This means when you do
> s = socket.tcp()
> s:setfd(fd)
> You actually get a "loose" fd, which is terrible if your application
> must handled hundreds or thousands of connections per minutes,
> you run VERY quickly into the OS fd limit.
> So you could use a small C lib to implement the OS close(fd) call
> then do socket.tcp(), getfd() on it to close it, then setfd(fd).
> This is not exactly intuitive.
>
> And anyway since socket.tcp() makes the object a tcp{master} it
> does not work anyway.
> To get it to work the only way is to do a "fake" connect() call to
> a port you know will not respond to change the class to tcp{client}.
> Like in this example:
> s = socket.tcp()
> s:connect("127.0.0.1", 12312)
> myclose(s:getfd())
> s:setfd(fd)
> data = s:recieve()
>
> This is extremely clunky and requires a C module that implements the OS
> close() call otherwise you get lost fds.

Lua function return types are not static, so socket.tcp() could return
a tcp{master} in the general case, and a tcp{client} if it is passed a
"serializable socket description". The caller knows if he's passing a
parameter or not, so checking the return type shouldn't even be
mandatory.

> I believe making threaded socket servers is a quite basic and common
> problem and thus this "solution" is not workable with it.
> Lanes is a really nice threading module, I like it a lot, but the
> inability to do network based stuff with it is annoying as hell,
> so as long as we do not get native threading in a single lua state
> we must find a workaround.
>
>> Now, if what we are talking about functionality that allows
>> you to obtain this OS-dependent socket handle/descriptor and
>> create another LuaSocket object from it, we can discuss what
>> is the best API for it. It is a valid scenario that I had
>> not foreseen, and I'd be happy to add it to the library.
>
> I do agree my patch is probably not the best or most beautiful,
> I did it only to resolve my problem, and it seems it is not only
> my problem :) But yes I do not code, nor care much I must confess,
> for Windows and a solution for it would be desirable too.
>
> However if windows does not use integer fds for its handles I am not
> sure it would work anyway. Can Lanes Lindas transfer something like that
> between states ? It only works for unix because fds are ints and so
> are transported just fine with Lindas.

I assume you can transport strings through Lindas. At the very least,
whatever a socket is on Windows (or any other platform) if you have it
in one thread, you can get it's address in memory, serialize the
address, send it, have the other thread deserialize the memory
address, and unreference it. IMHO this doesn't seem like a good
solution to circumvent Lanes inability to share resources between
threads.

> Maybe luasocket could keep an internal global list of all openned sockets
> with the fd/handle/... they refer to in the OS and then the multiple lua
> states could all use some luasocket API to access it via a simple number.
> But this would mean to make the global list threading resistant which is
> probably out of the scope of luasocket (and also an extremely ugly solution..)
>
> Well I guess I have written enough or too much already, good luck to
> those that read all of my bad english :)

The root of the problem is that a module is loaded twice in
independent Lua states, but it has no knowledge of that. Ideally,
passing a socket to another Lane would mean that the two instances of
the module have to be aware of each other, be able to pass data
structures from one to the other, even in the eventuality that one of
the instances is terminated (its parent state is closed).

This would make C modules writing a lot more complicated. Instead of
avoiding global variables to help multiple instanciation of a module,
you would have to make sure everything is global, and that userdata
are merely references to globally accessible structures. The C library
could provide some way to associate a serializable id to an object,
pass the id to another instance (in an independent state), and get the
object back (re-encapsulated in the new State).

The nice part is that it would make the work independent of LuaLanes,
and eventually compatible with all "multiple-independent-states" Lua
threading libraries. However I don't know the LuaSocket sources, and
such an adaptation could mean any amount of work between mere
adjustments to a complete rewrite.