Re: [ANN] Lua 5.2.1 (work1) now available

[Miles Bader <miles@...>]

Roberto Ierusalimschy <roberto@inf.puc-rio.br> writes:
>> >> Can you recommend any parameter changes to make the 5.2 garbage
>> >> collector more "5.1-like"?
>> >
>> > To make the collector more agressive you must increase its stepmul.
>> > By how much I do not know (see below).
>> 
>> I tried a bunch of values for "stepmul", from the default (200) up to
>> 10,000...  But none of the stepmul values I tried seemed to make much
>> difference -- Lua 5.2 never seemed able to keep the memory usage under
>> control (it still used over twice as much memory as 5.1.4 or LuaJIT,
>> and caused the system to start thrashing).
>
> May you try to reduce the pause?
>   collectgarbage("setpause", p)
> (For p=0 or 10 or 100...)

Ok, I tried this.

The summary:  helps a little for Lua 5.2, but also greatly increases
runtime.  Memory usage still never gets anywhere near Lua 5.1.4/LuaJIT
even with a value of "pause" that doubles runtime.

Unfortunately I think the dramatic slowdown makes "pause" not much of
a solution for this issue.  I also tried combining low values of
"pause" with increased values of "stepmul", but that didn't seem to
help.

[Lua 5.1.4 and LuaJIT become _much_ worse with "pause" below 100
though (like 10-20x slower).]

One thing about this particular test that might make it somewhat
unusual:  it allocates _very_ large strings (like 500MB), parses their
contents, and then drops all references to them.  Lua 5.1.4 very
quickly reclaims a huge amount of memory after these big strings
become unreferenced; printing out memory usage (using
collectgarbage"count") at various points shows many places where usage
in 5.1.4 drops from like "700MB used" to "2MB used", whereas the same
printouts in Lua 5.2 never shows any obvious large drops in usage.

Could this extreme "chunkiness" of allocation have some adverse impact
on 5.2 that it didn't on 5.1.4?

Thanks,

-Miles


Lua 5.2.0:

GC pause       Scene def CPU           Total CPU     Elapsed   MaxRSS
---------------------------------------------------------------------
       0  ... completely unusable ...
      10  248.7~1.4s (+4.4s)  256.1~1.4s (+4.7s)  271.2~4.5s  2,337MB
      25  177.1~0.8s (+4.7s)  184.7~0.8s (+5.0s)  199.2~3.8s  2,461MB
      50  151.4~1.1s (+4.5s)  158.9~1.1s (+4.7s)  176.7~5.0s  2,758MB
     100  140.6~1.3s (+4.9s)  148.9~1.3s (+5.4s) 193.1~29.1s  3,113MB
     200  134.1~0.9s (+8.0s)  143.9~1.0s (+9.0s)  237.9~7.5s  3,359MB


GC-pause = 100:

Lua variant             Scene def CPU           Total CPU     Elapsed   MaxRSS
------------------------------------------------------------------------------
luajit2.0.0-beta9  125.1~0.1s (+4.0s)  128.9~0.2s (+4.2s)  133.3~0.2s  1,309MB
lua5.1.4           153.6~1.9s (+2.3s)  157.6~1.9s (+2.5s)  160.4~2.0s  1,312MB
lua5.2.0           140.6~1.3s (+4.9s)  148.9~1.3s (+5.4s) 193.1~29.1s  3,113MB


GC-pause = 200 (default):

Lua variant             Scene def CPU           Total CPU     Elapsed   MaxRSS
------------------------------------------------------------------------------
luajit2.0.0-beta9  115.5~0.9s (+4.3s)  119.4~1.0s (+4.5s)  124.8~3.8s  1,626MB
lua5.1.4           134.9~2.3s (+2.5s)  138.9~2.4s (+2.7s)  141.9~2.6s  1,653MB
lua5.2.0           134.1~0.9s (+8.0s)  143.9~1.0s (+9.0s)  237.9~7.5s  3,359MB


-- 
Circus, n. A place where horses, ponies and elephants are permitted to see
men, women and children acting the fool.