Re: Lua hook and debugger implementation questions

[Dibyendu Majumdar <mobile@...>]

On 21 May 2016 at 10:29, Jean-Luc Jumpertz <jean-luc@celedev.eu> wrote:
> I’m not sure if your question is more about a  Lua debugger implementation in Lua or in C, but here are some answers for the C implementation case.

My implementation is in C. Fortunately I have my own modified VM in
Ravi so I have some freedom when it comes to implementation.

>
>> First - at present there isn't an ability to invoke the hook when an
>> error is thrown or caught. Is it possible / good idea to add support
>> for this? From a debugger perspective the throwing or catching of an
>> error is similar to function call / return.
>
> Break-on-error in the debugger can easily be achieved (without any change to Lua) if you control the launch of Lua code in the system using `lua_pcall`.
>
> The basic idea in this case  is to provide an error function to `lua_pcall` that directly calls the Lua hook in case of error. Here is a simplified version of the one I use in the CodeFlow IDE:
>
> static int pcallMessageHandler (lua_State* luaState)
> {
>    const char* errorString = lua_tostring(luaState, -1)];
>     // Do whatever is needed with errorString
>
>     if (interruptsOnError)
>     {
>         lua_Debug debugInfo;
>
>         if (lua_getstack(_luaState, 1, &debugInfo))
>         {
>             // debugInfo is initialized with a valid stack context
>             debugInfo.event = LUA_HOOKERRRUN;  // An additional define not conflicting with other hook values
>            debugHook(&debugInfo);
>         }
>     }
> }
>
> As Lua keeps the current error function by default for nested pcall invocations, defining the error function for the top level `pcall` is enough to have the break-on-error behavior in all cases.
>

Thank you for this suggestion. Is this a reliable approach - i.e. can
a user supply their own error routine in pcall and thereby change the
behaviour?

>> Secondly - what is the best way to implement 'step over' and 'step
>> out' actions in the debugger? A simple tracking of call / return
>> statements will fail if an error is thrown - i.e. a mechanism is
>> needed I would have thought to reset the tracking data when an error
>> is thrown that crosses the stack frame where the debugger was
>> originally paused.
>
> As Paul K mentioned, using stack level counters in the Lua hook can be a bit tricky. However, with a few additional hooks in the Lua core allowing the precise tracking of resume / yield calls, you can achieve very reliable step-over and step-out debugger actions, by having a separate counter for each Lua thread, at least while no error is thrown.
>
> Concerning he update of stack level counters in case  of errors, I haven’t implemented it yet, but you could try adding a couple of hooks in function `lua_pcallk`: the first one before the call to `luaD_pcall` for saving the current debugger context in a C stack variable; the second one after  `luaD_pcall` returns for restoring the debugger context in case of error.
>

I am thinking of a different approach - I am looking at whether I can
modify the line hook to provide the stack level by maintaining stack
level information in the CallInfo structure. Please see my reply to
Paul. Will this approach work?

Regards
Dibyendu