About line hook

["Tai Meng" <tmeng@...>]

Hi, I am fairly new to Lua and I am writing a simple debugger for it. While playing around with line hooks, I noticed that lua_sethook() does not initialize the line hook -- it is tracexec() in lvm.c that initializes the line hook when traceexec() is first called. Traceexec() is first called just before the first line of Lua code is executed. It then returns after initializing the hook, rather than executing the hook. The next time traceexec() is called (just before the second line of Lua code executes), the line hook is called because it has been initialized.

That meant the line hook does not start until the second line of Lua code. I found a couple of solutions to overcome this late initialization, one is to call luaG_inithooks( ) declared in ldebug.h, and the other is to include lstate.h and explicitly code in lua_state_pointer->hookinit = 1.

I wonder if there was a motivation for not letting the line hook execute before the first line of Lua code (could it have been an oversight?). I also wonder if one of the two solutions I found ( calling luaG_inithooks() vs. lua_state_pointer->hookinit = 1 ) is preferable to the other, and whether they are both safe (i.e. won't later cause something unexpected). 

Tai


-- for your reference... --

// lvm.c
static void traceexec (lua_State *L) {
  lu_byte mask = L->hookmask;
  if (mask > LUA_MASKLINE) {  /* instruction-hook set? */
    if (L->hookcount == 0) {
      resethookcount(L);
      luaD_callhook(L, LUA_HOOKCOUNT, -1);
      return;
    }
  }
  if (mask & LUA_MASKLINE) {
    CallInfo *ci = L->ci;
    Proto *p = ci_func(ci)->l.p;
    int newline = getline(p, pcRel(*ci->u.l.pc, p));
    if (!L->hookinit) {
      luaG_inithooks(L);
      return;
    }
    lua_assert(ci->state & CI_HASFRAME);
    if (pcRel(*ci->u.l.pc, p) == 0)  /* tracing may be starting now? */
      ci->u.l.savedpc = *ci->u.l.pc;  /* initialize `savedpc' */
    /* calls linehook when enters a new line or jumps back (loop) */
    if (*ci->u.l.pc <= ci->u.l.savedpc ||
        newline != getline(p, pcRel(ci->u.l.savedpc, p))) {
      luaD_callhook(L, LUA_HOOKLINE, newline);
      ci = L->ci;  /* previous call may reallocate `ci' */
    }
    ci->u.l.savedpc = *ci->u.l.pc;
  }
}

// ldebug.c
LUA_API int lua_sethook (lua_State *L, lua_Hook func, int mask, int count) {
  if (func == NULL || mask == 0) {  /* turn off hooks? */
    mask = 0;
    func = NULL;
  }
  L->hook = func;
  L->basehookcount = count;
  resethookcount(L);
  L->hookmask = cast(lu_byte, mask);
  L->hookinit = 0;
  return 1;
}