Re: Embedded lua and yielding to the application run loop (macOS)

[Jerome Vuarand <jerome.vuarand@...>]

On Mon, 17 Feb 2020 at 11:34, Aaron Magill <asmagill@icloud.com> wrote:
> So... I hit upon this hairbrained idea to "mimic" the macOS application runloop for a short period in a C function added to a lua module, defined like this:
>
>           static int hs_yield(lua_State *L) {
>               NSTimeInterval interval = (lua_type(L, 1) == LUA_TNUMBER) ? 0.001 : lua_tonumber(L, 1) ;

Isn't there a bug in the line above? Or is the ternary operator in
this language different than in C?

>               NSDate         *date    = [[NSDate date] dateByAddingTimeInterval:interval] ;
>
>               // a melding of code from gnustep's implementation of NSApplication's run and runUntilDate: methods
>               // this allows acting on events (hs.eventtap) and keys (hs.hotkey) as well as timers, etc.
>               BOOL   mayDoMore = YES ;
>               while (mayDoMore) {
>                   NSEvent *e = [NSApp nextEventMatchingMask:NSAnyEventMask
>                                                   untilDate:date
>                                                      inMode:NSDefaultRunLoopMode
>                                                     dequeue:YES] ;
>                   if (e) [NSApp sendEvent:e] ;
>
>                   mayDoMore = !([date timeIntervalSinceNow] <= 0.0) ;
>               }
>
>               return 0 ;
>           }
>
> So if I do the following in the Hammerspoon console (pasted in as one large chunk, which means its treated as a single string and also executed within a lua_pcall:
>
>           -- creates a grey rectangular box on the screen with the tet "**" in it
>           cv = hs.canvas.new{ x = 100, y = 100, h = 100, w = 100 }:show()
>           cv[#cv + 1] = { type = "rectangle", fillColor = { white = .1 } }
>           cv[#cv + 1] = { type = "text", text = "**", textSize = 75 }
>
>           st = false
>           bl = 0
>           dd = 0
>
>           -- every second, update the number displayed in the grey box
>           xx = hs.timer.doEvery(1, function()
>               dd = dd + 1
>               cv[2].text = tostring(dd)
>           end)
>
>           -- after 30 seconds, stop while loop below and clean up
>           yy = hs.timer.doAfter(30, function()
>               st = true   -- stop runaway while
>               xx:stop()   -- stop timer updating canvas
>               cv:delete() -- delete canvas
>           end)
>
>           zz = os.time()
>           while (not st) do
>               bl = bl + 1
>               hs_yield(n) -- defaults to 0.001 if n is nil
>           end
>           print(bl, dd, os.time() - zz)
>
> And this works as I had hoped -- the canvas is updated each second for 30 seconds and bl ends up being somewhere around 1478052.
>
> My question to the list: is this safe, or am I setting things up for a subtle catastrophe at some point?

This isn't safe on the Lua side of things.

> As I understand it, what's happening is that the lua interpreter is executing the while loop, and from inside the while loop, pauses for 0.001 seconds to see if an event is queued for the macOS application run loop; when there is, if the event has a lua function attached, it is invoked (from one of the timers) within a new lua_pcall, but before the original lua_pcall, which is executing the while loop, has finished.

This is not very clear to me (is it to you?). The key question is
whether the timer callbacks can only be called from
nextEventMatchingMask, on that same thread, or if the timer callbacks
are asynchronous. It is fine to have "recursive" lua_pcall-s, ie.
pcall function x, and have x pcall function y. But you cannot pcall
function x from thread 1, and pcall function y from thread 2 at the
same time.

> Is this safe, since each chunk of lua code is within its own lua_pcall? Are there lua commands I should make sure that hs_yield is never invoked within (haven't tried a for command iterator function yet, for example)?

lua_pcall doesn't protect against that situation. With a normal build
of Lua (see below for more) there are no lua commands you can use to
protect the state. You must use an external mutex.

So whether it is safe at all depends on the semantics of your timers
and your main loop event pump. Whether this can happen is a question
for experts of your OS APIs.

> Are there additional sanity checks I should make to ensure the integrity of the lua state and stack?

You can make a special build of Lua where you redefine the macros
lua_lock and lua_unlock. In these you can add code that will check a
flag is not set and set it (in lock) and clear it (in unlock). So if
two threads try to lock at the same time, you'll see the flag is
already set and it proves your code is unsafe.

> Figured I should ask here before releasing this into the core Hammerspoon code and let our users start banging on it.

To increase the chance of triggering the bug, you should decrease the
timeout in the main loop (0.001 seconds is a long time, try the
minimum possible value, ideally zero), and reduce the delay in your
timers so they trigger much more frequently. But to be entirely sure
you need to check the documentation for your timer API and your main
loop event polling API.