LuaJIT 1.1.5 issue/bug?

[Tim Mensch <tim-lua-l@...>]

I'm using LuaJIT 1.1.5 embedded in a Windows/Mac game SDK (Playground), 
and it's be working fine for me. One of the SDK users, though, pointed 
out a problem reported when using the Microsoft Application Verifier. 
(http://bit.ly/SalYW)

When run with the Application Verifier, I end up with a crash that is 
100% reproducible, though only after a relatively complex set-up. After 
some deep debugging, I came up with this "fix":

static Table *jit_createstate(lua_State *L, StkId arg, int nargs)
{
 Table *st;
 int i;
// If this function reallocates the stack, then arg, which points into 
the old stack, is invalidated
//  luaC_checkGC(L);
 st = luaH_new(L, nargs, COMSTATE_PREALLOC);
 for (i = 0; i < nargs; i++)
   setobj2t(L, luaH_setnum(L, st, i+1), arg+i);
 return st;
}

The luaC_checkGC() call can reallocate the stack to a completely new 
location; the Application Verifier may be forcing realloc() to always 
pick a new location? Not sure why the bug doesn't show up normally, 
otherwise.

In any event, this looks like it might be a real bug, since arg can 
become a dangling pointer. At least when the realloc() happens when 
jit_createstate() is called from luaJIT_run(), the next line crashes:

 Table *st = jit_createstate(L, func+1, L->top-(func+1));
 int status = jit_compile(L, func, st, 0); 

The crash happens on clvalue(func) in jit_compile(), since func is now 
pointing at the old stack location, which is now bad memory. I'm 
assuming that simply commenting out the offending luaC_checkGC() should 
be safe, so I don't need any immediate assistance, but if my analysis is 
correct, then this may be a real (if extremely infrequent!) bug, and not 
just a ghost caused by the verifier. Sorry if I've just missed something 
obvious.

LuaJIT is awesome, of course.

Tim



__________ Information from ESET NOD32 Antivirus, version of virus signature database 4969 (20100323) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com