lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

I'm working on a patched version of Lua 5.1 that adds a new "rope" subtype to strings, which allows postponing allocating and building concatenation operations until the final string's actually required. I've been able to get it working, and it's made quite a difference in performance. However, I've been dealing with some pretty bad memory leaks, and I'm not quite sure what's causing it.

I'm not super familiar with Lua's garbage collector code, but I've been learning it bit by bit while debugging this. When implementing the type, I made sure to add in GC code, following the same methods that other types like LUA_TPROTO use. It appears to work fine with basic concatenations, but as soon as a rope gets stored in a table for a little bit, running the garbage collector fails to free *some* of the memory. For example, this code uses about 78 MB of memory on my machine in the REPL, but after calling collectgarbage("collect") (in a new chunk, after t is out of scope), it only drops to 62.7 MB instead of ~1.5 MB:

local t = {} for i = 1, 1000000 do t[i] = "a" .. "b" end

The weird thing is that the leftovers do seem to be collected at the end - enabling LUA_USE_APICHECK doesn't complain about the allocated memory count being too high in the assert at lstate.c:122 - but they're not counted in the memory usage in collectgarbage("count"). My current theory is that the rope objects are somehow being dropped from the GC's linked list, but this doesn't match up with the fact that they're collected at the end - it might also be something somehow marking the objects even though they're not reachable.

I'd appreciate some advice on how to approach this issue, as at this point I'm completely lost on where the leak could be coming from. I've tried using valgrind, heaptrack, and Visual Studio, but none of these give me any useful information, and I'm not sure if I should try to patch Lua to look through the GC info somehow (and specifically what I should be looking at). I've posted the patch on top of Lua 5.1.5 if anyone wants to take a look at it: (the lgc.c portion starts on line 137). I'd greatly appreciate any help I can get.

-- JackMacWindows

Attachment: OpenPGP_0x1D99413F734AA894.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature
Description: OpenPGP digital signature