Re: Lua and preemptive scheduling for coroutines

[Leo Romanoff <romixlev@...>]

----- Ursprüngliche Message -----
> Von: Jose Luis Hidalgo <joseluis.hidalgo@gmail.com>
> An: Leo Romanoff <romixlev@yahoo.com>; Lua mailing list <lua-l@lists.lua.org>
> CC: 
> Gesendet: 12:43 Sonntag, 31.März 2013
> Betreff: Re: Lua and preemptive scheduling for coroutines
> 
> Hi All, I'm not sure if this helps... but anyway. I've been working on
> a coroutine manager, that allows to register C functions that can
> suspend lua coroutines, and C functions that can receive a lua
> function and create a new coroutine from it. I use to convert a C
> event based API to lua sequence where you can use "waitFor" like
> methods to wait until some condition/event is met to continue the
> execution.

Very cool! In fact, I was sure that this can be done if approached from C/C++ side. And it does not need any modifications to the Lua VM.

What I disliked in the existing approaches was the fact that C/C++ part was the main driver of the application.
But your approach can be used as a library that I use from Lua if I understand correctly. 
So, basically in Lua I can use new abstractions provided by this library to create threads and coroutines (using async) and to wait for certain events (using wait_for).
In a way, they replace/hide standard coroutine.create/yield/resume methods and provide more powerful abstractions build on top of it.

BTW, async is more or less coroutine.create + coroutine.resume() + some book keeping? Or is it more than that?

Another question: You do not support preemptive multtasking, do you? I.e. your async threads need to use wait_for or channel:read to yield, right?

> This is the header of the coroutinemanager:
> https://code.google.com/p/slb/source/browse/include/SLB3/extra/coroutinemanager.h

Looks pretty nice and clean.

> The usage is pretty simple, Inherit from CoroutineManager :
>   - override init_implementation to register your own functions
> (waitfor-like functions and async-like functions)
>   - optionally override event_implementation method if you want to
> monitor the creation/destruction/suspend of coroutines
>   - signal coroutines to wake them once they are ready to continue the execution
> 

One thing I'd like to have is the ability to provide a Lua code implementing parts of the logic that 
- decides which thread should be executed next
- defines new waitfor-like or async-like functions
- etc

I.e. I'd like to be able to "derive" from your CoroutineManager in Lua. Since it is not possible directly the idea is to have a class LuaBasedCoroutineManager which allows me to provide Lua code for different overridable methods. Then each such method could look like:

void LuaBasedCoroutineManager::methodX(args) {
    if(luaCodeForX != null) {
        // execute this lua code with provided args
    } else {
        // invoke default implementation (e.g. from the base class)
    }
}

If something like this would be possible, I could write my Lua apps without any need for writing a single line of C/C++ code. I only need to load your library. No makefiles, no need for C compilers, etc. No need for script developers to know C.

Is this feasible? What do you think?

> This is a basic example of usage of the coroutine manager (from the test suite):
> 
> https://code.google.com/p/slb/source/browse/test/test_coroutines.cc
> 
> 
> This is an example of a channel based comunication, where coroutines
> sync using channels (inspired by Go-lang channels, Erlang, and so on):
> 
> https://code.google.com/p/slb/source/browse/test/example_coroutinemanager.cc
> 
> That's probably the most complex example of the coroutine-manager

Very nice!  The channels example is very interesting.

While looking at your APIs and examples I couldn't understand if your wait_for constructs could wait for a specific event or events? Or are they simply a replacement for yield, which result in waiting for a resume call. I think being able to wait for a certain event from a set of alternatives or for all events from a certain set would be very powerful.

> The coroutinemanager is part of SLB3 (Simple Lua Binder, to wrap C++
> objects to lua), but it can be used alone. I haven't documented
> properly SLB3 yet, neither previous versions of SLB... but the code is
> used in our game engine on a daily basis, in fact the coroutine
> manager has been backported from the game engine to SLB .
 
OK. I briefly looked at the code and it seems to be small and self-contained enough to be used on its own.

It is a bit pity that the most interesting comments on this thread arrived after I have spent time on creating a working solution ;-) If you guys have had commented before, I would probably use one of your solutions, instead of reinventing my own from scratch. But on the other hand I learned a lot about Lua VM and coroutines.

Regards,
  Leo