Re: Stack overflow in luaO_pushvfstring

[Andrew Gierth <andrew@...>]

>>>>> "Yongheng" == Yongheng Chen <changochen1@gmail.com> writes:

 Yongheng> The original POC that can crash a default build lua ( just
 Yongheng> run make on Linux) is:

 Yongheng> function errfunc() end function test()
 Yongheng>     xpcall(test, coroutine.wrap(function() print(xpcall(test, errfunc)) end))
 Yongheng>         end coroutine.wrap(function() print(xpcall(test, errfunc)) end)()

So surely this, in lua_resume, can't possibly be right:

        L->nCcalls = getCcalls(from) + from->nci - L->nci - CSTACKCF;

We get here when the stack overflow error trips the first time and we
try and resume the wrapped coroutine which is supplied as the error
handling function. getCcalls(from) is correctly below CSTACKMARK, since
we're in stack error handling, but from->nci is large (~1900) reflecting
the large number of active stack frames. I get that _unused_ CallInfos
are already accounted for in getCcalls (so that it need only be checked
when new CallInfos are allocated), but here its adding a number
reflecting the _used_ CallInfos, which is surely wrong. (L->nci is 0
because the coroutine has no CallInfos yet.)

So we get into the coroutine with a large value (>2000) in L->nCcalls,
and so the coroutine itself proceeds to recurse _another_ ~1900 times
before tripping its own nested stack error, at which point the whole
thing repeats. Hence the stack overflows.

-- 
Andrew.