Re: C functions in Lua coroutines

[Eduardo Bart <edub4rt@...>]

> A quick Google search yielded this library which might be of interest:

    https://github.com/edubart/minicoro

    "Minicoro is single-file library for using asymmetric coroutines in C. The API is inspired by Lua coroutines but with C use in mind."

Author of minicoro here. Let's not forget about LuaCoco from Mike Pall, it is a patch that allows you to yield across C calls for Lua 5.1:

https://coco.luajit.org/ 

Minicoro is inspired by LuaCoco, but can be used with just C or any programming languages that have C interoperability. Bear in mind that you have to take more care with C stackful coroutines as they can consume resources (pre allocated stack space), moreover from the point of view of an optimizing C compiler, switching stack can trigger undefined behaviour. This issue is noticeable when you implement a coroutine scheduler in a multithread code that uses TLS (thread local storage), the C compiler may cache pointer to thread local storages in the stack, but if a coroutine resumes in a different thread, the cache will be become invalid (belonging to the previous thread) and you will have headaches. Overall, if you understand its limitations, stackful coroutines for C can be handy.

--

Eduardo Bart

Em ter., 28 de mar. de 2023 às 14:33, David Sicilia <dpsicilia@gmail.com> escreveu:

Currently there is no way to yield from C.

There are libraries (cross-platform to varying extents) that allow creating stackful coroutines (or "fibers" as they are sometimes called) in C or C++, which then allow yielding and resuming C/C++ functions.  For example, in C++ there is the boost::coroutine library.  I'm sure there are others for C.  A quick Google search yielded this library which might be of interest:

    https://github.com/edubart/minicoro

    "Minicoro is single-file library for using asymmetric coroutines in C. The API is inspired by Lua coroutines but with C use in mind."

(I have not looked at it in detail, so can't vouch for it, but looks relevant).

David

On Tue, Mar 28, 2023 at 2:29 AM Schmalzer, Lukas <Lukas.Schmalzer@trumpf.com> wrote:

Hello,

 

I don’t know if it was ever discussed, that when you call C functions in Lua coroutines, the whole Lua system is blocked, i.e., no coroutine can advance.

Currently there is no way to yield from C. This behavior is problematic for time critical applications, e.g., as in our application, the execution environment for machine sequences.

 

We implemented a way to yield from C, i.e., only the coroutine executing the long running C code is blocked.

As a solution, we create a coroutine in the interpreter (C code) for each coroutine in the Lua code. We applied the solution to the blocking C functions in “io” and “os” (liolib.c and loslib.c).

Of course, we are aware, that this cannot be a general solution, as existing code would not expect yields from these libraries. Maybe a separate library with functions to yield from C would be a good approach.

The yield from C feature makes most sense for custom, application specific C libraries.

I think that the possibility to yield from C would be a useful feature for the interpreter, because not the whole Lua system needs to block when a long running C function is executed.

Please let me know what you think about this feature.

 

Best regards,

Lukas