On Wed, Mar 28, 2012 at 4:09 PM, Graham Wakefield
<wakefield@mat.ucsb.edu> wrote:
I'm doing a similar thing wrapping libuv (which is what Luvit is built on) in a Lua module. All the potentially blocking calls, such as filesystem IO, become coroutine yield/resume when invoked from within a coroutine, making the code look very clean indeed.
Libuv is pretty nice I have to say, and offers everything you're looking for AFAICT (including working on Windows). It also doesn't have to take over the main thread loop, so can be integrated with mainloop-hungry GUI APIs.
I guess I should put this code somewhere for people to look at - but would need to clean it up first...
On Mar 28, 2012, at 4:56 AM, steve donovan wrote:
> On Wed, Mar 28, 2012 at 1:37 PM, Axel Kittenberger <
axkibe@gmail.com> wrote:
>> The more you work with immutables, the easier it gets.
>
> True, but using Lua we are restricted, and luvit is a similar kind of
> solution to the single-threaded issue as Node.js is for _javascript_.
>
> I must say that _coroutines_ make writing async-style code so much
> more pleasant. For instance, in this snippet that sets up a Windows
> named pipe server:
>
> winapi.make_pipe_server_async(function(f)
> while true do
> local res = f:read()
> if res == 'close' then break end
> f:write(res:upper())
> end
> print 'finis'
> end)
>
> the file object f is wrapped (a la Copas) so that it is not actually
> blocking on reads. Getting the write to be non-blocking continues to
> be an interesting exercise...
>
> Similar tricks are possible for making waiting-on-processes coroutine-friendly.
>
> steve d.
>
I first chose the "many threads with a global lock" architecture because it does not require any central scheduler or event loop and because it seemed very easy to use. Blocking sockets ? add a thread. Callback based API ? add a thread. GUI ? add a thread. You kind of understand the song here.
The work with many threads went fine for a while (it even worked with Qt) but at some point in maturing the code things started to become really nasty (deadlocks, crashing on signal reception) so I took the advice of some more experienced Lua users and moved to a single "poll" based event loop.
For this to work, I had to rewrite all the blocking parts, had to write my own Socket class and manage transitions between zeromq's event loop and Qt (when the GUI is used). I even had to write a poll API for avahi (which was hard to do).
That was a lot of work but I feel it was really worth it. Main benefits:
1. Writing C++ wrappers for non-blocking IO is not trivial but it is far easier then writing code to handle all the threading issues in a portable way.
2. Stability has greatly improved.
3. CPU usage is lower on idle.
4. Lua written code is easier to write for data consistency (you know that your code will not be interrupted unless you call "sleep" or wait on a socket).
5. You can use OpenMP or tons of threads by connecting them with Pipes or sockets which is easy (= no performance loss).
6. Having a single thread means that we do not have to worry about thread-specific issues such as creating/deleting GUI objects or calling OpenGL methods.
With multiple threads, the code quickly gets filled with things like this:
app:post(function()
win:delete()
end)
And it becomes really ugly.
Cheers,
Gaspard
