 |
lua-l archive |
|
|
||
|
I'm working on a patched version of Lua 5.1 that adds a new "rope" subtype to strings, which allows postponing allocating and building concatenation operations until the final string's actually required. I've been able to get it working, and it's made quite a difference in performance. However, I've been dealing with some pretty bad memory leaks, and I'm not quite sure what's causing it.
I'm not super familiar with Lua's garbage collector code, but I've been learning it bit by bit while debugging this. When implementing the type, I made sure to add in GC code, following the same methods that other types like LUA_TPROTO use. It appears to work fine with basic concatenations, but as soon as a rope gets stored in a table for a little bit, running the garbage collector fails to free *some* of the memory. For example, this code uses about 78 MB of memory on my machine in the REPL, but after calling collectgarbage("collect") (in a new chunk, after t is out of scope), it only drops to 62.7 MB instead of ~1.5 MB:
local t = {} for i = 1, 1000000 do t[i] = "a" .. "b" end
The weird thing is that the leftovers do seem to be collected at the end - enabling LUA_USE_APICHECK doesn't complain about the allocated memory count being too high in the assert at lstate.c:122 - but they're not counted in the memory usage in collectgarbage("count"). My current theory is that the rope objects are somehow being dropped from the GC's linked list, but this doesn't match up with the fact that they're collected at the end - it might also be something somehow marking the objects even though they're not reachable.
I'd appreciate some advice
on how to approach this issue, as at this point I'm completely
lost on where the leak could be coming from. I've tried using
valgrind, heaptrack, and Visual Studio, but none of these give
me any useful information, and I'm not sure if I should try to
patch Lua to look through the GC info somehow (and
specifically what I should be looking at). I've posted the
patch on top of Lua 5.1.5 if anyone wants to take a look at
it: https://gist.github.com/MCJack123/f1d6ca961260e93e806f89f4451e2c86
(the lgc.c portion starts on line 137). I'd greatly appreciate
any help I can get.
-- JackMacWindows
Attachment:
OpenPGP_0x1D99413F734AA894.asc
Description: OpenPGP public key
Attachment:
OpenPGP_signature
Description: OpenPGP digital signature