Re: Coroutine library scheduler

[Teto <mattator@...>]

Thanks for the many detailed answers. I will look into lumen to
replace my current code. You gave me ideas.

Once gain thanks a lot

2014-02-24 23:33 GMT+01:00 Andrew Starks <andrew.starks@trms.com>:
> On Mon, Feb 24, 2014 at 4:14 PM, Sean Conner <sean@conman.org> wrote:
>>
>> It was thus said that the Great John Hind once stated:
>> > I do not have a suggestion for anything that already exists, but I do
>> > think something really powerful could be done with a coroutine scheduler
>> > for Lua.
>> >
>> > For example, I would like to be able to say "yield this coroutine until
>> > some real-time criterion is met" ( yield(for 10 seconds), or yield(until
>> > packet received OR 20 seconds has elapsed) ).
>>
>>   I've written such code, to turn event based programming (using poll() or
>> epoll(), depending upon the underlying operating system [1].  One thing to
>> remember:  Lua coroutines are cooperative, not preemtive [2].  For the code
>> I've written, given that it's mostly network driven, I explicitely ignore
>> the fact that any coroutine could hog the CPU indefinitely (that is:  I have
>> to make sure I have no infinite loops---I'm not going to attempt to
>> implement preemptive threading).  But, any routine that could potentially
>> "block" now becomes a point of scheduling, which means, I have to supply my
>> own read() and write() calls (or even sleep()).
>>
>>   (For the record, I recently did a test and figured out that a coroutine on
>> 32 bit x86 system is around 900 bytes---much smaller and lighter than a
>> pthread)
>>
>> > But that coroutine scheduling idea is just so technically sweet ...
>>
>>   The scheduler I have is simple:  I'm either waiting for an event, or I'm
>> not.  As such, I have two lists, one of a list of coroutine waiting for an
>> event, and a list of coroutines that are ready to run (because their events
>> have arrived).  Simple round-robin scheduling.  I found no need for anything
>> more complicated than that. [3]
>>
>>   -spc (I can post code if anyone is interested)
>>
>> [1]     Unix variants, I should note.
>>
>> [2]     Yes, you can do preemtive Lua coroutine, if you want performace to
>>         drop like a stone.  To do it safely, you have to hook into the debug
>>         layer and trap either intructions (each routine to run for X
>>         instructions), lines, or functions.  I personally don't recomend it.
>>
>> [3]     Really, any more complicated than that, and you know what?  You're
>>         writing a kernel.
>>
>
>
>
> I thought I should follow up with more than "Lumen looks nice!"...
>
> Whenever I think about multi-threading Lua, my brain leaps to message
> passing patterns.
>
> Given that, llthreads + zmq or nanomsg + some kind of serializer /
> deserializer  = a pretty powerful tool set. Add a coroutine scheduler
> and you have a something that can handle events, idling, etc.
>
> That's kind of what Lumen is about. It has tasks, the ability to kill
> a thread, timeouts, pipes, etc.
>
> This is from the readme file:
>
> ```lua
>
>     local sched=require 'lumen.sched'
>
>     -- task receives signals
>     sched.run(function()
>         local waitd = {'an_event'}
>         while true do
>             local _, data = sched.wait(waitd)
>             print(data)
>         end
>     end)
>
>     -- task emits signals
>     sched.run(function()
>         for i = 1, 10 do
>             sched.signal('an_event', i)
>             sched.sleep(1)
>         end
>     end)
>
>     sched.loop()
>
> ```
>
> The tests folder is helpful as a tour guide, as well.
>
> Sometimes parts are a bit... out of sorts, but this is to be expected
> with something that is under development. It's *documented* (gasp)
> pretty well and its design seems very reasonable. It is based on
> SierraWirelsse's luasched[1].
>
> Also, it comes with baked in support for nixio, tcp/ip and possibly others.
>
> -Andrew
>
> [1] https://github.com/SierraWireless/luasched/
>