Re: tolua++ objects not garbage collected

[Tim Mensch <tim-lua-l@...>]

You're stepping the collector? And a complete GC fixes the problem? I'm going to guess that you're not stepping it enough.

I'm using tolua++ (with lots of modifications) in my game, and for the most part the problems that I've had are when tolua++ forgets a reference it should have kept rather than the other way around. I've had to add additional bindings between my C++ objects and Lua objects to ensure they share the same lifespan.

When I first started using the "step" instead of a full GC every frame, I found that the memory usage climbed as you describe. So now I step the collector three times per frame; for my usage and allocations, that seems to be enough. Try doing more than one step per frame if you can, and see if that helps.

The difference between allocating a table and allocating a tolua++ object is likely the fact that tolua++ caches the object in a hidden table that is marked to store values with weak references. That table is referenced by the metatable of each of the various objects as well, and every frame you're touching that table with new allocations, which might be invalidating the GC of the table values? Just a guess, really; I haven't dug into WHY I need to do it, but calling step three times makes it work for me. :)

tolua++ has actually been driving me nuts, and I plan to rip it out and replace it with something much cleaner (have you looked at the code?) as soon as I can. To be fair, I'm trying to get it to do things it wasn't designed to do. The only generic binding that seems to work really well is LuaBind, but it causes too much template code bloat.

Tim

On 5/4/2011 9:59 AM, Chris Redden wrote:

Hiya,

 

We’ve been using tolua++ for one of our projects and finding it an absolute joy to work with, very easy to use and understand. However we’ve recently hit a snag as our game gets further along, we’re finding a memory issue with tolua++ user objects and the lua GC. I tried to e-mail the tolua++ author at tolua@codenix.com but I got a rejection from their mailserver.

 

Anywhere we expect tolua++ objects to be garbage collected (such as a local Vector3 in a function), the GC seems to ignore the userdata pointer in the lua state every cycle, and at the same time constantly increase the GCThreshold until eventually our memory store runs out. The __gc metamethod is being called and tolua++ is calling the correct collector, allowing us to release our App side memory, but when I monitor the lua memory we get the above issue.

 

Example:

 

function update()

                local vec = Entity:GetPos()

end

 

This is the result of a basic script for the GC, each time the allocations hit the threshold, the threshold practically doubles.

 

Mem Usage: 213469,      Esti. Mem Usage: 195549,            GC Threshold: 391000,   GC Dept: 0,         GC State: pause

Mem Usage: 213593,      Esti. Mem Usage: 195549,            GC Threshold: 391000,   GC Dept: 0,         GC State: pause

Mem Usage: 213685,      Esti. Mem Usage: 195549,            GC Threshold: 391000,   GC Dept: 0,         GC State: pause

Mem Usage: 213841,      Esti. Mem Usage: 195549,            GC Threshold: 391000,   GC Dept: 0,         GC State: pause

Mem Usage: 213997,      Esti. Mem Usage: 195549,            GC Threshold: 391000,   GC Dept: 0,         GC State: pause

Mem Usage: 214025,      Esti. Mem Usage: 195549,            GC Threshold: 391000,   GC Dept: 0,         GC State: pause

...

Mem Usage: 390981,      Esti. Mem Usage: 195549,            GC Threshold: 391000,   GC Dept: 0,         GC State: pause

Mem Usage: 391009,      Esti. Mem Usage: 195549,            GC Threshold: 392033,   GC Dept: 9,         GC State: propagate

Mem Usage: 391037,      Esti. Mem Usage: 195549,            GC Threshold: 392033,   GC Dept: 9,         GC State: propagate

Mem Usage: 391065,      Esti. Mem Usage: 195549,            GC Threshold: 392033,   GC Dept: 9,         GC State: propagate

Mem Usage: 391093,      Esti. Mem Usage: 195549,            GC Threshold: 392033,   GC Dept: 9,         GC State: propagate

Mem Usage: 391121,      Esti. Mem Usage: 195549,            GC Threshold: 392033,   GC Dept: 9,         GC State: propagate

Mem Usage: 391149,      Esti. Mem Usage: 195549,            GC Threshold: 392033,   GC Dept: 9,         GC State: propagate

Mem Usage: 391177,      Esti. Mem Usage: 195549,            GC Threshold: 392033,   GC Dept: 9,         GC State: propagate

Mem Usage: 391205,      Esti. Mem Usage: 195549,            GC Threshold: 392033,   GC Dept: 9,         GC State: propagate

...

Mem Usage: 401733,      Esti. Mem Usage: 195549,            GC Threshold: 402393,   GC Dept: 129,    GC State: propagate

Mem Usage: 467297,      Esti. Mem Usage: 195549,            GC Threshold: 467297,   GC Dept: 64009,                GC State: propagate

Mem Usage: 467297,      Esti. Mem Usage: 394325,            GC Threshold: 467297,   GC Dept: 55845,                GC State: finalize

Mem Usage: 467325,      Esti. Mem Usage: 298325,            GC Threshold: 467325,   GC Dept: 47681,                GC State: finalize

Mem Usage: 467353,      Esti. Mem Usage: 205125,            GC Threshold: 410200,   GC Dept: 0,         GC State: pause

Mem Usage: 467381,      Esti. Mem Usage: 205125,            GC Threshold: 467381,   GC Dept: 48989,                GC State: propagate

Mem Usage: 467409,      Esti. Mem Usage: 205125,            GC Threshold: 467409,   GC Dept: 40825,                GC State: propagate

Mem Usage: 410093,      Esti. Mem Usage: 409425,            GC Threshold: 818800,   GC Dept: 0,         GC State: pause

Mem Usage: 410121,      Esti. Mem Usage: 409425,            GC Threshold: 818800,   GC Dept: 0,         GC State: pause

Mem Usage: 410149,      Esti. Mem Usage: 409425,            GC Threshold: 818800,   GC Dept: 0,         GC State: pause

Mem Usage: 410177,      Esti. Mem Usage: 409425,            GC Threshold: 818800,   GC Dept: 0,         GC State: pause

Mem Usage: 410205,      Esti. Mem Usage: 409425,            GC Threshold: 818800,   GC Dept: 0,         GC State: pause

 

We’ve not modified the GC in any way, and to make sure this wasn’t something general I did the same but creating an empty lua table every frame, the GC behaved correctly.

To add to the confusion, if I force a full garbage collection every frame, the issue goes away and the object is collected correctly.

 

Any ideas what could be prevent the lua GC from picking up tolua++ objects correctly?

 

Regards,

Christopher Redden