RE: Yielding from hooks

[RLake@...]

> From what I can tell luaD_callhook is the gate every hook call is going
through.

I believe that is true.

> This call unconditionally leads to luaD_call.

Where? luaD_callhook calls the hook directly at line 173:

(*hook)(L, &ar),

> Any attempt to yield from within luaD_call will fail as
> it increments nCcalls making lua_yield bail if this
> value is > 0. I have just tried to create a C hook
> function and had the same failure regardless whether
> it is being set from C or from Lua.

I don't think I was being very clear. You would have to
install the hook from C, using the Lua API. The C hook
could then lua_call a lua function, but the lua function
itself could not call coroutine.yield; it would have to
be the hook function which called lua_yield. If you
install the hook function, whether a Lua closure or a
C closure, using the debug.sethook interface, it will
not be able to yield.

> The main application for yielding hooks was, as you have suggested,
> to implement non-cooperative multitasking.
> I don't see why this would be incompatible with the use of coroutines.
> All we are talking about is which function calls lua_yield.
> In any case, inside hook or not, the yield would occur within
> the same coroutine context,

That is the problem; it would not be the same coroutine context.
Say you have a scheduler, which starts up various threads. One of
these threads, call it T, decides to create a coroutine as a generator (G).
So the parent of T is the scheduler; if T yields, it returns to the
scheduler.
But the parent of G is T; when G yields, it returns its result to T. If
a yielding hook is active while G is running, it will yield to T, not
to the scheduler. That would probably be disastruous, which is why I think
that yielding hooks are incompatible with the use of coroutines.

> We will survive without it, but I would like to hear your
> ideas on how to make it work.

The basic idea is to let C functions tailcall Lua functions.
If that were allowed, then hookf (in ldblib.c) could tailcall
the hook function instead of lua_call'ing it, and there would be
no problem with yields except for the problem I mentioned above.

Something similar to this would work for metamethod calls as well,
albeit through a slightly different mechanism. In almost all cases,
the result of an opcode which invokes a metamethod is the value
returned by the metamethod, so if instead of lua_call'ing the
metamethod, the opcode mutated itself into an OP_CALL followed by
an OP_MOVE, the metamethod would be able to yield. I cannot see
any good reason why a metamethod would want to yeild, actually, but
it would also allow hook functions invoked within the metamethod
to yield.