Re: Lua coroutines to async/await feature

[Russell Haley <russ.haley@...>]

On Wed, Nov 23, 2016 at 11:11 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
> On 23 November 2016 at 02:38, William Ahern <william@25thandclement.com> wrote:
>> On Wed, Nov 23, 2016 at 12:59:30AM +0000, Dibyendu Majumdar wrote:
>>> This is just something I have been wondering about. I personally have
>>> not yet found a use case for Lua's coroutines - the main reason being
>>> that the resume has to be explicit. What would be more useful is if
>>> the coroutine semantics could be harnessed to create an async/await
>>> type solution as in C#. That is, can a solution be found to
>>> automatically resume the coroutine when an event associated with a
>>> 'yield' completes. As Lua is single threaded so the solution needs to
>>> resume the coroutine within the same OS thread.
>>
>> For some definition of "automatic", this is actually quite trivial.
>> Obviously it cannot be too automatic because Lua has a contract with C code.
>> But this can be implemented quite transparently from the perspective of
>> Lua-script space. The biggest caveat is that Lua objects are multithread
>> safe.
>>
>
> Thanks for the detailed reply - much appreciated. Did you mean above
> that Lua objects are not multi-thread safe?
>
>>> I haven't really tried solving this but I thought it might be a useful
>>> enhancement to Lua if a solution could be found. I imagine that a
>>> background OS thread will be needed to monitor completion of events
>>> and then some coordination with the VM will be necessary to resume
>>> suspended coroutines, maybe whenever the VM does an operation that
>>> requires system resources. In some ways the 'go' language works like
>>> this - i.e. there is a background scheduler that swaps in suspended
>>> goroutines to the thread whenever execution would otherwise pause.
>>
>> Goroutines work more like Lua coroutines than you'd think. While the kernel
>> threads that goroutines execute on are preemptively scheduled, AFAIU the Go
>> goroutine scheduler is cooperative. Control is usually passed between
>> goroutines when doing reads and writs to a channel. The Go compiler also
>> inserts checkpoints in compiled code, usually at function call points, to
>> support garbage collection. Scheduling might also occur at those points.
>>
>> So while goroutines _feel_ preemptively scheduled, they're actually
>> cooperative scheduled underneath the hood. It's why loops in Go code that
>> don't do any I/O, polling, or message passing can hog the entire kernel
>> thread and stop progress of other goroutines as well as the garbage
>> collector.
>>
>> Lua's coroutines can be used in much the same way, you just need to
>> implement a scheduler much like Go does. Lua gives you the tools to
>> implement those patterns, you just need to build an application framework
>> around to make it feel transparent.
>>
>
> I had a quick look at your project cqueues that presumably does this.
> Would it be a good idea to port this to libuv so that Windows can be
> supported as well (I understand cqueues only works on UNIX like
> environments)?

libuv is a different beast that does the same thing but not
specifically to lua, it was written for the node.js project if I
remember correctly (https://en.wikipedia.org/wiki/Libuv). Mr. Aherns
solution is very elegant because it blends yielding and direct
connection to the os calls. I would love to hear more from Mr. Ahern
about this!

Have you looked at luvit (https://luvit.io/install.html)?

Russ