Behavior of lua_resume() in Lua 5.4

[Sergei Zhirikov <sfzhi@...>]

Hi,

I have been looking through the Lua 5.4 manual to see what's new. One small detail has raised some questions and concerns.
This is what lua_resume() function looks like in Lua 5.4:

int lua_resume (lua_State *L, lua_State *from, int nargs, int *nresults);

Compared to Lua 5.3, the function has acquired a new parameter - nresults. The explanation of its meaning in the manual seems pretty clear at the first sight. But once you think about it for a minute, it turns out not so clear at all, because it raises more questions than it answers. The most obvious one is: when will the value of *nresults be different from the stack size (as returned by lua_gettop())? Then, whenever that happens, the stack will contain some values besides the ones passed by the coroutine when yielding. What are those values? And most importantly, what am I supposed to do with them? Regarding the last question, the manual says:

"To resume a coroutine, you clear its stack, push only the values to be passed as results from yield, and then call lua_resume."

This sentence has, by the way, not changed compared to Lua 5.3. It made perfect sense back then, but now, in combination with this nresults thing, it doesn't make much sense to me. Why do I get something returned on the stack that I don't know what it is, cannot do anything useful with it and should throw away in the end anyway?

I have performed some experiments with the new lua_resume() is several scenarios. What I have observed so far, is that lua_resume() can return "extraneous" values (that is *nresults < when lua_gettop()) only when the coroutine has yielded from within a C function using lua_yieldk() with a continuation function. The extraneous elements on the stack then reflect the stack of the yielded C function. Are my observations correct? Is that the only case?

Elsewhere in the manual, the description of lua_yieldk() says the following:

The "continuation function receives the same stack from the previous function, with the n results removed and replaced by the arguments passed to lua_resume."

That is is quite clear and makes perfect sense. But it seems to imply that I *must not* remove the extraneous values, returned by lua_resulme(), from the stack before the next call to lua_resume(), otherwise the continuation function will not be able to receive them as documented. That is then in direct contradiction with the sentence quoted earlier that instructs to clear the stack. Is this a documentation error or am I misunderstanding something?

In general, I am wondering what prompted this change to lua_resume()? What would be an example of a use case when having those extraneous values returned would be useful? In other words, what was wrong with the way Lua 5.3 worked?

One way or another, this change to lua_resume() means that every developer who uses it must now re-examine each and every call to see how the change affects the stack balance and what needs to be done to rectify possible issues. In some cases it may even require to add another field to the internal state of the C component/application to save the *nresults value until just before the next call to lua_resume(). Personally, I am not looking forward to that exercise.
To simplify migrating existing code to Lua 5.4 (IMO significantly) I would like to suggest the following:
    * Make the nresults parameter optional, that is allow for it to be NULL.
    * If nresults is not NULL, lua_resume() will behave exactly as it does now.
    * If nresults is NULL, lua_resume() will behave the same as it did in Lua 5.3 (and earlier).

Assuming there are use cases when the new behavior has added value, I nevertheless strongly suspect that it will not be useful in most cases. If that's true, the suggested change will be useful not only to simplify migration, but also to make writing new code a little bit easier.

Last, but not least: the new lua_resume() doesn't appear to always behave as described in the manual. When the coroutine returns (rather than yields) the value of *nresults seems to always be equal to what lua_gettop() returns. That means it counts not only the values returned by the coroutine, but also those that have been at the bottom on the stack since before the coroutine was started. I can't imagine that to be the intended behavior. It looks very much like a bug to me.

--
Thanks and regards,
Sergei.