Re: Buglet: Unexpected hook calls

[Rici Lake <lua@...>]

Rici Lake wrote:
> Also, call and return hooks would be a lot more useful if they
> carried an indication of the number of actual arguments and
> return values (resp.). I'll see if I can find an elegant way of
> doing this.

I took a look at this again, and I think I have a simple
solution. Basically, I'd like to be able to use call
and return hooks to trace the arguments and return values
from functions, including CFunctions.

The current debug interface is a little unsatisfactory
In the case of C functions, the call hook is entered with
the callframe set up perfectly: the arguments extend from
[L->base, L->top). (But see below.)

In the case of Lua functions, the call hook is entered
with the adjusted arguments in the correct place. However,
L->top is already set to ci->top, so it's not really
possible to know where the arguments end, even by
examining local names, since any locals defined in line 1
of the function will be apparently visible, along with
name parameters.

If the hook call (lines 299-303 in ldo.c) were done three
lines earlier, just after ci->nresults = nresults; then
L->top would still point at either the last supplied or
the last needed fixedarg (whichever is less), which would
be reasonable.

There remains a single minor issue: the hook function
hookf in ldblib.c leaves a table on the stack which shows
up at the end of the callframe, since the hook function
runs inside the hooked callframe. This could be easily fixed
by adding:

  lua_replace(L, -2);

after the rawget(L, -1); at line 212.

For return hooks, the situation is slightly simpler since it
doesn't differ between C and Lua functions. The hook is called
with L-top pointing just after the last returned value, but it
has no way of knowing where the first returned value is. However,
callrethooks (in ldo.c) does know where the first returned
value is; it's passed that as an argument. If the first call
to luaD_callhook (at line 333 in ldo.c) were changed from
   luaD_callhook(L, LUA_HOOKRET, -1);
to:
   luaD_callhook(L, LUA_HOOKRET, firstResult - L->base);
then the hook function would be able to use its second argument
to know where to start looking for return values.

Again, this would require the minor change to hookf in ldblib.c
to be usable by hook functions written in Lua.

The remaining problem is that there is no debug interface which
makes it possible to view varargs. It should be simple enough
to provide one new debug interface:

  lua_getvararg (lua_State *L, int level, int argindex);

and a corresponding Lua binding in the debug. library.
I don't think that lua_setvararg is absolutely necessary,
but it would be nice to be able to see the varargs.

I could produce a patch of all of that if it would be useful.

R.