Re: LuaJIT roadmap 2008

[Mike Pall <mikelu-0802@...>]

Stephen Kellett wrote:
> Will it be possible to use the debug API to implement debuggers and 
> profilers that work with LuaJIT 2.x? I'm thinking of both perspecitives: 
> from Lua and from C.

The reflection APIs already work (I needed working tracebacks),
the debug hooks are not implemented, yet. Not because it's
particularly difficult, but because I'm not sure how this will
work out in conjunction with the other parts of the VM and still
be fast enough in all use cases.

The idea is to patch the instruction dispatch table at runtime
when hooks are enabled (I'm already doing this for the recording
phase). This means it's zero-cost when no hooks are enabled. But
the instruction table is in the global state and not the thread
state. So a single thread that has hooks enabled affects the
performance of all other threads (the hook handlers are not
called for them, but dispatch is still slowed down).

And it gets worse -- how should the recording phase or running
compiled code be handled when a hook is set? The easy thing is to
cop out and just stay in interpreter mode. But e.g. sandboxed
environments set an instruction hook with a high count to catch
runaway scripts. These would never enjoy the full benefits of a
JIT compiler. Just attaching a debugger enables all hooks, too.
This might slow down things enough to make running the app
unfeasible. And let me not talk about the difficulties of
enabling a hook in a signal handler while some machine code is
already running.

Let's put it that way: the Lua debug API is rather low-level.
This is certainly the way to go for a minimal API. But it's used
for a variety of purposes: interrupting execution with Ctrl-C,
catching runaway scripts and simulating breakpoints for a
debugger and so on. These have very different characteristics.

A higher level debug API could give more hints to the VM. It's a
real waste of resources to have the instruction hook enabled all
the time when all you're doing is catching runaway scripts (an
OS-level timer with some logic for calling a synchronous handler
would be better). Catching all thrown errors and some other
important events is impossible without patching the source. And
how many people have gone through all the hoops required to
reliably implement breakpoints in a Lua debugger? I know of at
least three ...

I'm sure you have your own share of experiences with this issue,
too. Right now I have no complete plan to change or extend the
debug API. Not even a complete requirements analysis. But I guess
I'll be forced to dive into this topic sooner or later.

--Mike