Re: Heap use after free in luaD_call

[Roberto Ierusalimschy <roberto@...>]

> Getting somewhere. An object (table) is ending up on the grayagain list
> (after the atomic phase of a generational step) while still new; it's
> still in that list when sweepgen sees it, which clears it to white
> without removing it from the list. A subsequent barrier call then sees
> it as white and turns it gray, adding it to the (empty) gray list, at
> which point everything _else_ that was on grayagain just got dropped on
> the floor, and this includes the main thread.

Another small step:

Up to the barrier, all was going on as expected.

The idea is that, after sweepgen clears it to white without removing
it from the gray list, a call to correctgraylist will remove it from
the gray list. (sweepgen cannot remove it because we need to traverse
the gray list to remove an element.)

However, still inside sweepgen, if a thread is collected, it has to
close its upvalues. This moves the value from the stack being
collected to the upvalue, and that needs a barrier (the one you
mentioned!). So, this barrier sees the object still in an inconsistent
state, between having its color cleared by sweepgen and it being
removed from the gray list by correctgraylist. Then, caos follows,
as you described.

-- Roberto