Re: Lua 5.4.0-rc5 segfault in low memory conditions

[Andrew Gierth <andrew@...>]

>>>>> "Andrew" == Andrew Gierth <andrew@tao11.riddles.org.uk> writes:

 Andrew> So my question is this:

 Andrew> The (original) test case basically does a ton of parsing in
 Andrew> which a whole lot of prototypes get generated and added to
 Andrew> other prototypes, in the context of building up one single
 Andrew> return statement.

 Andrew> Where in this are those prototypes supposed to be reached from
 Andrew> the GC? i.e. what's the intended path by which the GC is
 Andrew> supposed to find them?

Answering my own question: the top-level closure is supposed to be on
the stack (and indeed it is), and everything else should be reached
recursively from that.

But (this is while stopped in a deeply nested part of the parser, but
examining the state of the luaY_parser frame):

(gdb) print funcstate.f
$27 = (Proto *) 0x801614200

(gdb) print *((LClosure*)L->stack[5].val.value_.p)
$30 = {next = 0x80162e170, tt = 6 '\006', marked = 36 '$', nupvalues = 1 '\001', gclist = 0x0, p = 0x801614200, upvals = {0x0}}

So here's the closure on the stack whose prototype (p) points to the
top-level prototype created by luaY_parser. Notice that marked=36, which
is BLACK | G_OLD.

But:

(gdb) print *((LClosure*)L->stack[5].val.value_.p)->p
$31 = {next = 0x801621d00, tt = 10 '\n', marked = 16 '\020', numparams = 0 '\000', is_vararg = 1 '\001', maxstacksize = 2 '\002', 
  sizeupvalues = 4, sizek = 0, sizecode = 4, sizelineinfo = 4, sizep = 4, sizelocvars = 0, sizeabslineinfo = 0, linedefined = 0, 
  lastlinedefined = 0, k = 0x0, code = 0x801667020, p = 0x801651b40, upvalues = 0x801621d80, 
  lineinfo = 0x801668020 "\001", '\245' <repeats 39 times>, "\001&", abslineinfo = 0x0, locvars = 0x0, source = 0x80162e170, 
  gclist = 0xa5a5a5a5a5a5a5a5}

Here's the Proto that the closure points to, and it has marked = 16
(i.e. WHITE1). So here we have a classic black->white pointer scenario,
and none of the prototypes nested under this one will be scanned because
everything stops at the black closure.

So the bug is here in luaY_parser:

  funcstate.f = cl->p = luaF_newproto(L);
  /* needs another barrier here */
  funcstate.f->source = luaS_new(L, name);  /* create and anchor TString */
  luaC_objbarrier(L, funcstate.f, funcstate.f->source);

The GC forced by the memory failure happened inside luaF_newproto, and
resulted in the closure "cl" being marked both BLACK and OLD. Without a
barrier at the marked location, we have cl (black) pointing at the new
proto (white) and everything blows up later as a result.

-- 
Andrew.