Re: Forcing coroutine.resume to call the yielding function again

But the "trick" has no warranty to work in C: once you've past over the point of execution where you called a function (here lua_yield()) there's no warranty at all that you can return back to the point where you started the function calling it, because the environment is now different: the state of variables/registers/parameters is lost, and there's still NOTHING left explicitly in your code that allows you to come back and resurrect the initial state at start of the function in which you called lua_yield (at end of your example function, and possibly even after its termination when you call lua_yield as a trailing return).

Really, think about recreating your C function using an EXPLICIT loop and use the value that MUST be returned from lua_yield() and should be the value passed to coroutine.resume(value) in Lua. Returning at any other point of execution in C will just give unpredicable result and forcibly ignores the value given to coroutine.resume(value).

In summary you C function should do something like:

my_cfunction() {

do {

//...

} while (lua_yield());

}

where it explicitly uses the value passed in Lua (here as a boolean, tested by the "while" loop, and that can be passed in Lua as "coroutine.resume(true)" to restart the loop in the function, or "coroutine.resume(false)" to terminate it). You can pass any value in "coroutine.resume(value)" to control how your C function should behave.

But remember that coroutine.resume(...) is not supposed to restart any function but only continue its execution, exactly in its current context and flow, just after the point where lua_yield() returns.

Modifying the VM to do something else will break many uses inclujding in existing libraries where lua_yield() must return immediately after its call and use its value.

Remember that you can use lua_yield inside a sequence of instructions that is not supposed to be interrupted in the middle like in:

my_cfunction() {

do_something();

lua_yield();

do_something_else();

}

If you change that behavior in the VM to restart the C function, "do_something_else()" will never be called and you'll loop indefinitely on executing "do_something()".

So your approach is definitely broken. It's not the way to go. Use an explicit loop in C, and you solve the problem CLEANLY without changing anything in the Lua VM.

Le lun. 19 août 2019 à 16:08, Andreas Falkenhahn <andreas@falkenhahn.com> a écrit :

I think it is possible, you just need the right trick ;) I've now come up with something that seems to do the job: The key to success was realizing that it is perfectly legal for Lua hooks to yield. Since Lua hooks can be called before every instruction I can just use this insight to yield before OP_CALL instead of during it.

In other words, I just needed to add some code right to the spot where Lua will execute LUA_MASKCOUNT hooks: The code to be added checks if the next instruction is OP_CALL and it also checks if the C function to be executed is one of those few special C functions that should be able to yield and be called again on resume. If it is one of those functions, I'm preparing the stack and call the respective function manually and then I'm calling lua_yield() to yield control back to the callee. Lua now thinks a hook has yielded which is perfectly legal. Et voila, once resume() is called it will now start resuming execution right at the OP_CALL instruction that I've intercepted via a pseudo-hook. All VM states preserved. Seems to work just fine.

I'll have to do some more tests but AFAICS it does exactly what I need and it also seems safe to do so because it makes use of the Lua feature that hooks are allowed to yield so I can just yield before OP_CALL, manually run the C function and then have resume() continue at OP_CALL precisely.

On 19.08.2019 at 01:23 Philippe Verdy wrote:

> What you want is simply impossible to do in C: your testfunct uses
> a "return lua_yield(...)" (basically a trailing call that a C
> compiler may eventually optimize into a jump in some
> platform-specific conditions, but not always like in Lua). That
> lua_yield() calls is still in C, and the C stack pointer will return
> to the point where this call was done, i.e. still inside the
> "testunct" function where it will return (an optimizing compiler may
> eventually have resolved it as a jump (after popping the current
> stack frame of "testfunc" and repolacing it with the stack frame for
> "lua_yeild" just before dong the jump. The return address of the
> "testunction()" call is still in the C stack and not modified, but
> it points to the instruction AFTER the call to testfunc; there's no
> way to restart the function from the beginning, simply because
> there's no warranty that the stackframe inside the function
> "testfunc" is still there and valid (it may have already been
> destroyed and replaced by the stackframe for lua_yield(...).

> Yielding coroutines is not supposed to perform such loop implicitly
> (and tricking the Lua VM will not help you) : you have to write the
> loop explicitly yourself in C, around the call to lua_yield() inside
> the "testfunct" function, in order to allow it to loop, and use a
> continuation condition for your loop, that may depend on the value
> returned by lua_yield(), i.e. the value passed by the
> coroutine.resume(value), or on the value of an external variable...

> Coroutines are just pausing and always resuming **after** at the
> point where they yielded, but **never before that point**.

> Le dim. 18 août 2019 à 18:13, Andreas Falkenhahn
> <andreas@falkenhahn.com> a écrit :

> I have a C function which looks like this:
>
> static int testfunc(lua_State *L)
> {
> static int flag = 0;
>
> if(!flag) {
> flag = 1;
> printf("First call\n");
> return lua_yield(L, 0);
> } else {
> printf("Second call\n");
> }
>
> return 0;
> }
>
> It is called from a coroutine like this:
>
> co = coroutine.create(function()
> print("foo")
> testfunc()
> print("bar")
> end)
>
> coroutine.resume(co)
> coroutine.resume(co)
>
> Normally, the second call to coroutine.resume() will continue executing the code at
>
> print("bar")
>
> because testfunc() yields. However, instead of doing that, I'd
> like resume() to resume code execution at testfunc() instead of print("bar").
>
> So I've hacked lua_yield() to return -2 instead of -1 and then in
> lvm.c/OP_CALL I'm checking for -2 and, if it's found, decrement the
> PC stored in "savedpc" to make lua_resume() run OP_CALL again. The code in lvm.c looks like this:
>
> ...
> } else if (firstResult > L->top) { /* yield? */
> lua_assert(L->ci->state == (CI_C | CI_YIELD));
> (L->ci - 1)->u.l.savedpc = (firstResult == L->top + 2) ? pc - 1 : pc;
> (L->ci - 1)->state = CI_SAVEDPC;
> return NULL;
> }
> ...
>
> However, it doesn't work as expected because on resume, OP_CALL
> doesn't seem to be able to find "testfunc", presumably because the
> result of the OP_GETGLOBAL preceding OP_CALL isn't in the VM registers any more.
>
> When I decrement the PC by 2 instead of 1, everything works fine
> because OP_GETGLOBAL is called then to resolve the "testfunc"
> reference so that OP_CALL can find it. But this is confusing me a
> little because I thought that yielding/resuming would preserve the
> complete VM states so that it should be possible to force the VM to
> just re-execute the OP_CALL that was responsible for yielding but apparently that's not the case.
>
> So is there any other way to force the VM to re-run the function
> that yielded when resuming code execution on a coroutine?
>
> Note that I'm still on Lua 5.0.
>
> --
> Best regards,
> Andreas Falkenhahn mailto:andreas@falkenhahn.com
>
>
>

--
Best regards,
Andreas Falkenhahn mailto:andreas@falkenhahn.com