Re: Embedded lua and yielding to the application run loop (macOS)

[Aaron Magill <asmagill@...>]

On Mon Feb 17 19:09:20 GMT 2020, Oliver <oschmidt-mailinglists at gmx.de> wrote:


> I would execute time consuming tasks on separate system threads to keep the GUI
> thread responsive. This is also possible with Lua. Is this an option for you?

Desgin decisions that predate my joining the project put the lua interpreter on the
same thread as the user interface and application run loop. Can't do much about
it now with a fundamental re-write.

> Another option could be to have the time comsuming task within a coroutine that
> yields periodically. The coroutine would be called and continued in an event
> queue timer. With this you don't have to handle a local event queue loop.

First let me say that I don't have much experience with co-routines... it's a lack
I've been meaning to address, but so far haven't had a need that couldn't be
addressed another way or the time to do it on my own.

But if I'm understanding co-routines correctly, they still require some lua code (or
C code using the lua API) to actually invoke `coroutine.resume` on them for processing
to continue. This won't work for us (at least in the general case) because
the user interface, event queue, and lua interpreter share the main thread --
in Hammerspoon, as long as the lua interpeter is doing something, nothing else on
the main thread can.

> Special care has to be taken in the implementation of the the callbacks in the
> event run loop. On which lua_State do they operate? It could be the same
> lua_State than the lua_State that calls hs_yield, or it could also be a separate
> lua_State created be lua_newthread (i.e. coroutine).

Everything in Hamemrspoon uses the same lua_State -- the one created when Hammerspoon
first started (or a reload occurs, which destroys the state and creates a new one).

That's going to be a problem, isn't it, if someone does try to invoke
hs_yield from within a coroutine... is checking lua_isyieldable == 0 sufficient
to make sure we're not in a coroutine, or is there a better test?

> ... The event queue callback
> handling should keep the lua stack balanced and also ensure that there is enough
> space on the lua stack. Anyway all this should be happening in the Hammerspoon
> codebase, but perhaps you should add an extra call to lua_checkstack in your c
> function hs_yield since the Hammespoon probably doen't expect that there is
> another function running on the lua_State where the event queue callbacks are
> called. This would not be necessary if Hammerspoon uses a separate lua_State for
> the callbacks. There could be other pitfalls in the Hammerspoon codebase. You
> should ask the Hammerspoon developers or have a closer look on the Hammerspoon
> implementation.

Stack guards and tests for this were recently added but I agree additional checks
couldn't hurt, and this may inadvertantly cause the stack to grow, temporarily,
in ways it hasn't before, so thanks for the reminder!

> You should also consider that this solution doesn't nest very well: if there is
> at some point the "local event queue loop" that dispatches events and tries to
> handle some background operation and the dispatched events also go into such
> kind of "local event queue loop" (could also be the same) the first one has to
> wait until the second one finshes. But perhaps this is not the case in your use
> case.

I know, but without a fundamental rewrite, this isn't going to change...

--
Aaron