Re: Forcing coroutine.resume to call the yielding function again

[Andreas Falkenhahn <andreas@...>]

I think it is possible, you just need the right trick ;) I've now come up with something that seems to do the job: The key to success was realizing that it is perfectly legal for Lua hooks to yield. Since Lua hooks can be called before every instruction I can just use this insight to yield before OP_CALL instead of during it.

In other words, I just needed to add some code right to the spot where Lua will execute LUA_MASKCOUNT hooks: The code to be added checks if the next instruction is OP_CALL and it also checks if the C function to be executed is one of those few special C functions that should be able to yield and be called again on resume. If it is one of those functions, I'm preparing the stack and call the respective function manually and then I'm calling lua_yield() to yield control back to the callee. Lua now thinks a hook has yielded which is perfectly legal. Et voila, once resume() is called it will now start resuming execution right at the OP_CALL instruction that I've intercepted via a pseudo-hook. All VM states preserved. Seems to work just fine.

I'll have to do some more tests but AFAICS it does exactly what I need and it also seems safe to do so because it makes use of the Lua feature that hooks are allowed to yield so I can just yield before OP_CALL, manually run the C function and then have resume() continue at OP_CALL precisely.

On 19.08.2019 at 01:23 Philippe Verdy wrote:

> What you want is simply impossible to do in C: your testfunct uses
> a "return lua_yield(...)" (basically a trailing call that a C
> compiler may eventually optimize into a jump in some
> platform-specific conditions, but not always like in Lua). That
> lua_yield() calls is still in C, and the C stack pointer will return
> to the point where this call was done, i.e. still inside the
> "testunct" function where it will return (an optimizing compiler may
> eventually have resolved it as a jump (after popping the current
> stack frame of "testfunc" and repolacing it with the stack frame for
> "lua_yeild" just before dong the jump. The return address of the
> "testunction()" call is still in the C stack and not modified, but
> it points to the instruction AFTER the call to testfunc; there's no
> way to restart the function from the beginning, simply because
> there's no warranty that the stackframe inside the function
> "testfunc" is still there and valid (it may have already been
> destroyed and replaced by the stackframe for lua_yield(...).


> Yielding coroutines is not supposed to perform such loop implicitly
> (and tricking the Lua VM will not help you) : you have to write the
> loop explicitly yourself in C, around the call to lua_yield() inside
> the "testfunct" function, in order to allow it to loop, and use a
> continuation condition for your loop, that may depend on the value
> returned by lua_yield(), i.e. the value passed by the
> coroutine.resume(value), or on the value of an external variable...


> Coroutines are just pausing and always resuming **after** at the
> point where they yielded, but **never before that point**.


> Le dim. 18 août 2019 à 18:13, Andreas Falkenhahn
> <andreas@falkenhahn.com> a écrit :

> I have a C function which looks like this:
>  
>          static int testfunc(lua_State *L)
>          {
>                  static int flag = 0;
>  
>                  if(!flag) {
>                          flag = 1;
>                          printf("First call\n"); 
>                          return lua_yield(L, 0);
>                  } else {
>                          printf("Second call\n");
>                  }
>  
>                  return 0;
>          }
>  
>  It is called from a coroutine like this:
>  
>          co = coroutine.create(function()
>                  print("foo")
>                  testfunc()
>                  print("bar")
>          end)
>  
>          coroutine.resume(co)
>          coroutine.resume(co)
>  
>  Normally, the second call to coroutine.resume() will continue executing the code at
>  
>          print("bar")
>  
>  because testfunc() yields. However, instead of doing that, I'd
> like resume() to resume code execution at testfunc() instead of print("bar").
>  
>  So I've hacked lua_yield() to return -2 instead of -1 and then in
> lvm.c/OP_CALL I'm checking for -2 and, if it's found, decrement the
> PC stored in "savedpc" to make lua_resume() run OP_CALL again. The code in lvm.c looks like this:
>  
>          ...
>            } else if (firstResult > L->top) {  /* yield? */
>                  lua_assert(L->ci->state == (CI_C | CI_YIELD));
>                  (L->ci - 1)->u.l.savedpc = (firstResult == L->top + 2) ? pc - 1 : pc;
>                  (L->ci - 1)->state = CI_SAVEDPC;
>                  return NULL;
>            }
>          ...
>  
>  However, it doesn't work as expected because on resume, OP_CALL
> doesn't seem to be able to find "testfunc", presumably because the
> result of the OP_GETGLOBAL preceding OP_CALL isn't in the VM registers any more.
>  
>  When I decrement the PC by 2 instead of 1, everything works fine
> because OP_GETGLOBAL is called then to resolve the "testfunc"
> reference so that OP_CALL can find it. But this is confusing me a
> little because I thought that yielding/resuming would preserve the
> complete VM states so that it should be possible to force the VM to
> just re-execute the OP_CALL that was responsible for yielding but apparently that's not the case.
>  
>  So is there any other way to force the VM to re-run the function
> that yielded when resuming code execution on a coroutine?
>  
>  Note that I'm still on Lua 5.0.
>  
>  -- 
>  Best regards,
>   Andreas Falkenhahn                          mailto:andreas@falkenhahn.com
>  
>  
>  


-- 
Best regards,
 Andreas Falkenhahn                            mailto:andreas@falkenhahn.com