Re: Lua hook and debugger implementation questions

[Jean-Luc Jumpertz <jean-luc@...>]

Hi Dibyendu,

I’m not sure if your question is more about a  Lua debugger implementation in Lua or in C, but here are some answers for the C implementation case.

> First - at present there isn't an ability to invoke the hook when an
> error is thrown or caught. Is it possible / good idea to add support
> for this? From a debugger perspective the throwing or catching of an
> error is similar to function call / return.

Break-on-error in the debugger can easily be achieved (without any change to Lua) if you control the launch of Lua code in the system using `lua_pcall`.

The basic idea in this case  is to provide an error function to `lua_pcall` that directly calls the Lua hook in case of error. Here is a simplified version of the one I use in the CodeFlow IDE:

static int pcallMessageHandler (lua_State* luaState)
{
   const char* errorString = lua_tostring(luaState, -1)];
    // Do whatever is needed with errorString

    if (interruptsOnError)
    {
        lua_Debug debugInfo;
        
        if (lua_getstack(_luaState, 1, &debugInfo))
        {
            // debugInfo is initialized with a valid stack context
            debugInfo.event = LUA_HOOKERRRUN;  // An additional define not conflicting with other hook values
           debugHook(&debugInfo);
        }
    }
}

As Lua keeps the current error function by default for nested pcall invocations, defining the error function for the top level `pcall` is enough to have the break-on-error behavior in all cases.

> Secondly - what is the best way to implement 'step over' and 'step
> out' actions in the debugger? A simple tracking of call / return
> statements will fail if an error is thrown - i.e. a mechanism is
> needed I would have thought to reset the tracking data when an error
> is thrown that crosses the stack frame where the debugger was
> originally paused.

As Paul K mentioned, using stack level counters in the Lua hook can be a bit tricky. However, with a few additional hooks in the Lua core allowing the precise tracking of resume / yield calls, you can achieve very reliable step-over and step-out debugger actions, by having a separate counter for each Lua thread, at least while no error is thrown.

Concerning he update of stack level counters in case  of errors, I haven’t implemented it yet, but you could try adding a couple of hooks in function `lua_pcallk`: the first one before the call to `luaD_pcall` for saving the current debugger context in a C stack variable; the second one after  `luaD_pcall` returns for restoring the debugger context in case of error.

Hope this helps.

Jean-Luc