Re: Embedded lua and yielding to the application run loop (macOS)

[Oliver <oschmidt-mailinglists@...>]

Hi.

On 17.02.20 12:34, Aaron Magill wrote:
> I am a contributor to Hammerspoon, which uses an embeded lua 5.3 interpreter to allow users to programatically access a fair amount of the macOS API for things like window management, hoteys, etc. and I've been looking at ways to make it seem more responsive when the lua interpreter is actually doing something slow...

I would execute time consuming tasks on separate system threads to keep the GUI
thread responsive. This is also possible with Lua. Is this an option for you?

Another option could be to have the time comsuming task within a coroutine that
yields periodically. The coroutine would be called and continued in an event
queue timer. With this you don't have to handle a local event queue loop.

> As I understand it, when things like a timer fires and we want to invoke a lua callback, what happens is an event is queued for the macOS runloop and when the runloop isn't doing anything else, it checks its queue and pops an event off and handles it. In the case of one of our timers, the block attached to the queued event uses `lua_pcall` to call a stored function and the lua interpreter executes the lua function.

> So... I hit upon this hairbrained idea to "mimic" the macOS application runloop for a short period in a C function added to a lua module, defined like this:
>           zz = os.time()
>           while (not st) do
>               bl = bl + 1
>               hs_yield(n) -- defaults to 0.001 if n is nil
>           end
>           print(bl, dd, os.time() - zz)
>
> My question to the list: is this safe, or am I setting things up for a subtle catastrophe at some point?

In principle this should work, since everything is happening on the same thread.

Special care has to be taken in the implementation of the the callbacks in the
event run loop. On which lua_State do they operate? It could be the same
lua_State than the lua_State that calls hs_yield, or it could also be a separate
lua_State created be lua_newthread (i.e. coroutine). The event queue callback
handling should keep the lua stack balanced and also ensure that there is enough
space on the lua stack. Anyway all this should be happening in the Hammerspoon
codebase, but perhaps you should add an extra call to lua_checkstack in your c
function hs_yield since the Hammespoon probably doen't expect that there is
another function running on the lua_State where the event queue callbacks are
called. This would not be necessary if Hammerspoon uses a separate lua_State for
the callbacks. There could be other pitfalls in the Hammerspoon codebase. You
should ask the Hammerspoon developers or have a closer look on the Hammerspoon
implementation.

You should also consider that this solution doesn't nest very well: if there is
at some point the "local event queue loop" that dispatches events and tries to
handle some background operation and the dispatched events also go into such
kind of "local event queue loop" (could also be the same) the first one has to
wait until the second one finshes. But perhaps this is not the case in your use
case.

Best regards,
Oliver