GC param tuning (was: Re: LUA_GCSTEP)

["Adam D. Moss" <adam@...>]

Jasmin Patry wrote:
> > I am afraid we missed that. Can you send again?  We are eager for
> > feedback about GC parameters.
>
> Here is the change we made to luaC_step (we introduced the parameter
> "alpha"):
>
>     void luaC_step (lua_State *L) {
>       static const int alpha = 8; /* should be a factor > 1, larger =
> more aggressive GC, default = 2 */
>       global_State *g = G(L);
>       l_mem lim = (g->nblocks - (g->GCthreshold - GCSTEPSIZE)) * alpha;
>       do {
>         lim = singlestep(L, lim);
>         if (g->gcstate == GCSfinalize && g->tmudata == NULL)
>           break;  /* do not start new collection */
>       } while (lim > 0);
>       g->GCthreshold = g->nblocks + GCSTEPSIZE - lim/alpha;
>       lua_assert((long)g->nblocks + (long)GCSTEPSIZE >= lim/alpha);
>     }
>
> Note that this is against 5.1w0.

Interesting -- unless I've applied some patches to my 5.1w2
which I've forgotten about, w2's luaC_step() function is
algorithmically quite similar (where it #defines a
value called STEPMUL which is equivilent to your 'alpha').

One obvious difference is that you do the equivilent of
  g->GCthreshold = g->totalbytes + GCSTEPSIZE - lim/STEPMUL;
while w2 does merely
  g->GCthreshold = g->totalbytes + GCSTEPSIZE;

w2 also does a:
  g->GCthreshold = 2*g->estimate;
and you do not.

Your default STEPMUL is 8 while 5.1w2's is 2.

Now, I've been fiddling with these differences and these parameters.
At the same STEPMUL, your version is rather more effective at capping
the peak memory usage.  The '- lim/STEPMUL' difference does not seem
significant for this.

The g->GCthreshold = 2*g->estimate line is significant and has a
poor effect on peak memory usage.  Removing this line from w2's
luaC_step() function seems to make the two versions perform
very similarly w.r.t. peak memory usage.

Some figures for my app with the w2 function with the
2*g->estimate line removed:
STEPMUL    peakmem     FPS(speed)
2          3253K       107
4          3669K       108
7          3383K       106
8          2975K       108
9          2924K       107
10         2558K       108
11         2626K       106
12         2490K       106 (good and reproducable)
13         3169K       104
16         3065K       108
32         4123K        93
The curve is surprisingly reproducable, and indicates that
the relationship between STEPMUL and peakmem is not actually
monotonic.

For an interesting comparison, here is the
curve with the 2*g->estimate line intact (i.e. w2's default):
STEPMUL    peakmem     FPS(speed)
2          4532K       109
4          4934K       109
8          4626K       109
12         6000K+      110  (strange but reproducable)
16         4712K       109
32         4898K       109
Again quite reproducable, this curve has less deviation
(with a bizarre exception) but a considerably higher peakmem.

Finally, here's the w2 luaC_step() function with
the 2*g->estimate line removed and '- lim/STEPMUL' added
in the threshold update:
STEPMUL    peakmem     FPS(speed)
2          3607K       109
4          3758K       107
8          3205K       107
12         2600K       108
16         3077K       105
32         4132K        95

In general, FPS(speed) isn't greatly affected by
STEPMUL in a logical way, except in extreme circumstances.

This is only one application, and the data, although
reproducable, has some curious spots, but I'll leap
to the following recommendations for a good peakmem:
1) STEPMUL somewhere between 10 and 12
2) luaC_step() as in 5.1w2, but with this line removed or improved:
   g->GCthreshold = 2*g->estimate;
   (from a quick test it looks like 'g->GCthreshold = g->estimate'
   works as well as removing the line)

This yields (for my app):
STEPMUL    peakmem     FPS(speed)
12         2490K       106

... versus this for the default 5.1w2 func/settings:
STEPMUL    peakmem     FPS(speed)
2          4532K       109

Finally, here's the same statistic for lua 5.0.2 !
STEPMUL    peakmem     FPS(speed)
n/a        4480K       111

Regards,
--Adam
--
Adam D. Moss   . ,,^^   adam@gimp.org   http://www.foxbox.org/   co:3