Re: Crash Analysis : Erroneous finalizer called during tail call leads to heap buffer overflow

[Roberto Ierusalimschy <roberto@...>]

> Recently, my friends and I found an interesting crash from the latest Lua
> Interpreter. Using address sanitizer, we figured out that heap buffer
> overflow crash. To find the exact cause of the crash and to provide better
> patch guidance, we analyzed the crash more precisely.
> 
> [...]

Many thanks for the report and the analysis.


> - Patch Suggestion
> 
> Although the reason for the crash is simple, it was hard to patch. We have
> to change lots of code to handle this special case - finalizer being called
> during luaD_pretailcall function. To solve the problem, we devised 3
> different patches.

Couldn't we just move the stack/GC check to just before anything
is changed?

       int fsize = p->maxstacksize;  /* frame size */
       int nfixparams = p->numparams;
       int i;
+      checkstackGCp(L, fsize - delta, func);
       ci->func -= delta;  /* restore 'func' (if vararg) */
       for (i = 0; i < narg1; i++)  /* move down function and arguments */
         setobjs2s(L, ci->func + i, func + i);
-      checkstackGC(L, fsize);
       func = ci->func;  /* moved-down function */
       for (; narg1 <= nfixparams; narg1++)
         setnilvalue(s2v(func + narg1));  /* complete missing arguments */

The given size (fsize - delta) in this case is conservative. We could
give a more precise value, but I am not sure it is worth the extra
time to compute it.

-- Roberto