[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Change in GC behaviour -- Ever-increasing memory usage
- From: Pedro Tammela <pctammela@...>
- Date: Tue, 29 Jan 2019 19:04:34 -0200
> > I came across a supposed leak[1,2] with some C code. I think I distilled
> > the behaviour that causes problems into the attached Lua program and
> > would now like to get some input on "which code is to blame".
> > I noticed that the problem occurs with Lua 5.3, but does not occur with
> > Lua 5.1 or 5.2. Thus, I could do a Git bisect based on the GitHub mirror
> > of the Lua source code. The result is that commit  introduced the
> > problem.
> > My main question now is: What to do about this?
> > Some more information about the program: The library in question
> > provides bindings to GObject-based C libraries. To support callbacks, it
> > has to luaL_ref() the callback function and it will then luaL_unref()
> > later from the __gc metamethod of some of the involved object.
> > This is just what the attached program does: In a tight loop, it creates
> > a table, luaL_ref()s it and arranges for a call to luaL_unref() later.
> > Apparently the code can now create references faster than they are released.
> Lua controls de pace of its GC like this: during each cycle, it tries to
> estimate how much memory it is actually using. Then, after it completes
> the cycle, it waits for its memory use to grow by some percentage
> (typically 100%) before starting a new cycle.
> Because the program uses finalizers to release most of its memory, it
> needs two GC cycles to collect that memory. For objects with finalizers,
> Lua knows about that, and therefore these objects do not enter in
> that previous computation (how much memory is actually in use).
> However, Lua knows nothing about references. For Lua, all objects
> in the 'register' table are non garbage. So, by the end of a GC cycle,
> it looks like it is using all that memory, so it wait a little more
> for the next cycle. With this waiting, more objects will be in the
> registry by the end of the next cycle, and there is a vicious cycle.
> In my tests, Lua 5.1 presented the same problem. I am not sure why
> Lua 5.2 does not.
> -- Roberto
I believe this issue happens in 'normal' tables as well.
I've managed to reproduce this issue in both Lua 5.3 and Lua 5.4-work2
and could isolate this to a simple C code. The memory allocation
pattern in both version is pretty different, probably thanks to the
new GC in 5.4.
This code  will grow the memory steadily and will exhaust the
computer memory given enough iterations.
In my 8GB machine, this happened in 11 seconds when compiled with 5.3.
 both show the memory allocation pattern using massif from valgrind.