Re: Hooks, tailcalls and tracing LuaJIT

[Mike Pall <mikelu-1104@...>]

Geoff Leyland wrote:
> Has anyone got any bright ideas about how to detect tail calls
> or returns from the debug interface in LuaJIT?  Any chance that
> LuaJIT could report tail returns like Lua does (without
> compromising performance)?  Or that it could break the debug
> interface and add a hook argument for the "call" action that
> tells you whether the call is a tail call?

Lua 5.2 has an extra LUA_HOOKTAILCALL event. I can't use that for
LuaJIT, but I could pass the info in the line number. E.g. -1 for
regular calls and -2 for tail calls. Not sure if this breaks
anything in other debuggers. The pseudo-line number would only be
accessible from a C hook, not a Lua hook set with debug.sethook().

But note that the Lua debug interface is rather brittle, anyway.
It mixes too many different concerns, it's not designed to be
extensible, it works at the wrong abstraction level and it's
incomplete (doesn't report yields, errors). It may work for simple
tasks, but a full-blown debugger or trace analyzer needs way too
much guesswork and workarounds. Even setting a simple breakpoint
is excessively complicated.

In other words: design a new and better API for debuggers/profilers
and I might implement it. Or investigate ways to use existing
debug protocols (stepping through bytecode in GDB -- why not?).

--Mike