Re: Garbage Collector on Constrained System

[Ralph Hempel <rhempel@...>]

Rici Lake wrote:

> You can tune the garbage collector as described in the reference manual:
> http://www.lua.org/manual/5.1/manual.html#pdf-collectgarbage

I read this first, and have some confusion. As I understand it, the
setpause tells the garbage collector to run when the current memory
usage is some percentage of the usage the last time the collector ran.

The default of 200% (2) means memory use has to double before the
collector runs, which is fine if you have a huge heap. On my system
with 52K of heap, if I was using 26K the last time the collector
ran, it would not run again until I was out of memory.
> The key settings are setpause and setstepmul; I've found that in tight 
> loops with a lot of allocation of small objects (usually strings) and 
> where there are large tables of static reference data, you need to make 
> the garbage collector more aggressive; some experimentation may be 
> necessary to come up with a better tuning in your environment. 
> Generally, I adjust the pause to between 150 and 175, instead of the 
> default 200, but even smaller values are plausible. (Those are 
> percentages, so 200 actually means 2.0).

I am using 110 for now, so I'm on the right track, even if it's very
aggressive.

> The consequence of basing the gc's behaviour on the amount of memory 
> actually used is that gc tuning can be suboptimal if there is a lot
> of static data; transient objects can pile up until they reach the size 
> of the used heap at the end of the previous gc cycle (by default). So if 
> you have 128 megabytes of stuff which never changes, gc will be deferred 
> at the end of every gc cycle until 128 megabytes of transient objects 
> have built up. At that point it will start collecting garbage at a rate 
> roughly approximating twice the allocation rate (by default), but if the 
> allocations are smallish, quite a few can build up before each gc step, 
> and in extreme cases, memory usage can even continue to increase for a 
> while. In some ways, it might be better if there a way of setting an 
> expected heap floor, to compensate for such situations, so that the 
> pause would be based on a formula like p * (usage - floor) instead of 
> just p * usage. That might be even harder to tune, though.

Agreed. I'll keep using the aggressive settings and think about ways to
force GC to happen more frequently.

Thanks for the ideas,

Ralph