 |
lua-l archive |
|
|
||
|
I'm embarrassed to admit I didn't know about the step parameter; I'd been passing zero.
AND...I've added a lot to the game since I last verified that Lua wasn't growing uncontrollably, and it is growing again. :( Luckily my game has short levels, and I can do a full collection between times. I don't really have time before release to figure out what's leaking, and I suspect it has to do with the way tolua++ interacts with the way I want to do things. Just now I put in a few "heavy-handed" changes that force a lot of things to be deallocated between levels, and now it grows much more slowly.
But it still seems to grow, which might mean I need to nuke the entire Lua state from time to time. I'll keep an eye on it. And as I said before, I can't wait to rip the tolua++ code out, which will remove a lot of complexity and make it much more obvious to me when an allocation is happening, and where it's being stored.
Tim
On 5/4/2011 11:03 AM, Chris Redden wrote:I wasn't stepping the collector before, just relying on the standard GC logic to pick it up. What step size do you use when you step? Wouldn't stepping once with 3x that value produce the same result as stepping 3x?
Hmm, probably not as I recall the __gc metamethod doesn't cause the object to get cleaned until the following cycle.
- Christopher Redden
On 4 May 2011 17:55, Tim Mensch <tim-lua-l@bitgems.com> wrote:
You're stepping the collector? And a complete GC fixes the problem? I'm going to guess that you're not stepping it enough.
I'm using tolua++ (with lots of modifications) in my game, and for the most part the problems that I've had are when tolua++ forgets a reference it should have kept rather than the other way around. I've had to add additional bindings between my C++ objects and Lua objects to ensure they share the same lifespan.
When I first started using the "step" instead of a full GC every frame, I found that the memory usage climbed as you describe. So now I step the collector three times per frame; for my usage and allocations, that seems to be enough. Try doing more than one step per frame if you can, and see if that helps.
The difference between allocating a table and allocating a tolua++ object is likely the fact that tolua++ caches the object in a hidden table that is marked to store values with weak references. That table is referenced by the metatable of each of the various objects as well, and every frame you're touching that table with new allocations, which might be invalidating the GC of the table values? Just a guess, really; I haven't dug into WHY I need to do it, but calling step three times makes it work for me. :)
tolua++ has actually been driving me nuts, and I plan to rip it out and replace it with something much cleaner (have you looked at the code?) as soon as I can. To be fair, I'm trying to get it to do things it wasn't designed to do. The only generic binding that seems to work really well is LuaBind, but it causes too much template code bloat.
Tim
On 5/4/2011 9:59 AM, Chris Redden wrote:Hiya,
We’ve been using tolua++ for one of our projects and finding it an absolute joy to work with, very easy to use and understand. However we’ve recently hit a snag as our game gets further along, we’re finding a memory issue with tolua++ user objects and the lua GC. I tried to e-mail the tolua++ author at tolua@codenix.com but I got a rejection from their mailserver.
Anywhere we expect tolua++ objects to be garbage collected (such as a local Vector3 in a function), the GC seems to ignore the userdata pointer in the lua state every cycle, and at the same time constantly increase the GCThreshold until eventually our memory store runs out. The __gc metamethod is being called and tolua++ is calling the correct collector, allowing us to release our App side memory, but when I monitor the lua memory we get the above issue.
Example:
function update()
local vec = Entity:GetPos()
end
This is the result of a basic script for the GC, each time the allocations hit the threshold, the threshold practically doubles.
Mem Usage: 213469, Esti. Mem Usage: 195549, GC Threshold: 391000, GC Dept: 0, GC State: pause
Mem Usage: 213593, Esti. Mem Usage: 195549, GC Threshold: 391000, GC Dept: 0, GC State: pause
Mem Usage: 213685, Esti. Mem Usage: 195549, GC Threshold: 391000, GC Dept: 0, GC State: pause
Mem Usage: 213841, Esti. Mem Usage: 195549, GC Threshold: 391000, GC Dept: 0, GC State: pause
Mem Usage: 213997, Esti. Mem Usage: 195549, GC Threshold: 391000, GC Dept: 0, GC State: pause
Mem Usage: 214025, Esti. Mem Usage: 195549, GC Threshold: 391000, GC Dept: 0, GC State: pause
...
Mem Usage: 390981, Esti. Mem Usage: 195549, GC Threshold: 391000, GC Dept: 0, GC State: pause
Mem Usage: 391009, Esti. Mem Usage: 195549, GC Threshold: 392033, GC Dept: 9, GC State: propagate
Mem Usage: 391037, Esti. Mem Usage: 195549, GC Threshold: 392033, GC Dept: 9, GC State: propagate
Mem Usage: 391065, Esti. Mem Usage: 195549, GC Threshold: 392033, GC Dept: 9, GC State: propagate
Mem Usage: 391093, Esti. Mem Usage: 195549, GC Threshold: 392033, GC Dept: 9, GC State: propagate
Mem Usage: 391121, Esti. Mem Usage: 195549, GC Threshold: 392033, GC Dept: 9, GC State: propagate
Mem Usage: 391149, Esti. Mem Usage: 195549, GC Threshold: 392033, GC Dept: 9, GC State: propagate
Mem Usage: 391177, Esti. Mem Usage: 195549, GC Threshold: 392033, GC Dept: 9, GC State: propagate
Mem Usage: 391205, Esti. Mem Usage: 195549, GC Threshold: 392033, GC Dept: 9, GC State: propagate
...
Mem Usage: 401733, Esti. Mem Usage: 195549, GC Threshold: 402393, GC Dept: 129, GC State: propagate
Mem Usage: 467297, Esti. Mem Usage: 195549, GC Threshold: 467297, GC Dept: 64009, GC State: propagate
Mem Usage: 467297, Esti. Mem Usage: 394325, GC Threshold: 467297, GC Dept: 55845, GC State: finalize
Mem Usage: 467325, Esti. Mem Usage: 298325, GC Threshold: 467325, GC Dept: 47681, GC State: finalize
Mem Usage: 467353, Esti. Mem Usage: 205125, GC Threshold: 410200, GC Dept: 0, GC State: pause
Mem Usage: 467381, Esti. Mem Usage: 205125, GC Threshold: 467381, GC Dept: 48989, GC State: propagate
Mem Usage: 467409, Esti. Mem Usage: 205125, GC Threshold: 467409, GC Dept: 40825, GC State: propagate
Mem Usage: 410093, Esti. Mem Usage: 409425, GC Threshold: 818800, GC Dept: 0, GC State: pause
Mem Usage: 410121, Esti. Mem Usage: 409425, GC Threshold: 818800, GC Dept: 0, GC State: pause
Mem Usage: 410149, Esti. Mem Usage: 409425, GC Threshold: 818800, GC Dept: 0, GC State: pause
Mem Usage: 410177, Esti. Mem Usage: 409425, GC Threshold: 818800, GC Dept: 0, GC State: pause
Mem Usage: 410205, Esti. Mem Usage: 409425, GC Threshold: 818800, GC Dept: 0, GC State: pause
We’ve not modified the GC in any way, and to make sure this wasn’t something general I did the same but creating an empty lua table every frame, the GC behaved correctly.
To add to the confusion, if I force a full garbage collection every frame, the issue goes away and the object is collected correctly.
Any ideas what could be prevent the lua GC from picking up tolua++ objects correctly?
Regards,
Christopher Redden