How to use generational gc [was Re: A review of changes between 5.1 and 5.2-work3]

lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

Subject: How to use generational gc [was Re: A review of changes between 5.1 and 5.2-work3]
From: "Juris Kalnins" <juris@...>
Date: Mon, 24 May 2010 15:50:46 +0300

On Fri, 21 May 2010 21:00:13 +0300, Roberto Ierusalimschy<roberto@inf.puc-rio.br> wrote:

That will depend on the size of your new generations and the frequency
of major collections. (In an optimistic setting these could keep
latency low...)

[..]


I would like to have feedback on this, mostly from "real" programs
that do lots of allocation. (Simply call collectgarbage"gen" to
set generational mode.)

I tried it with small test program, because merging work3 into "real"codebase

will take some time.

The usecase I'm trying to test is Lua program with static, non-changingdata thatis much larger than the runtime data. I wanted to see whether enabling"gen"mode will reduce peak memory usage. I thought it should, because theamount ofdata in youngest generation is very small, so it should get collectedoften.

This seems not to be the case.
Also, running time is virtually the same in both modes.

I don't really understand how it works, but I'm confused by this conditionin the

"generationalcollection":
    if (g->totalbytes > g->lastmajormem/100 * g->gcpause)

Shouldn't the frequency and step size of minor collections be independentof the

size of older generation?





Here is the test program:
================================================================================

collectgarbage "stop"

local function g()
    local ret = {}
    for i=1,100 do ret[i] = {a=1} end
    return ret
end
local b = {}
for i=1,100 do b[i]=g() end

local t = {}

collectgarbage "restart"

collectgarbage "collect"
collectgarbage "collect"

--enable/disable "gen" mode
-- collectgarbage "gen"

local n = 0
for i=1,100000 do
    local s = "n" .. i
    t[n] = s
    if n == 4 then n = 0 else n = n + 1 end
--enable/disable "step" on each alloc
--    collectgarbage "step"
end

collectgarbage "collect"
collectgarbage "collect"

================================================================================

Here are results of running it under valgrind --tool=massif, with the
commented out lines enabled/disabled

Summary of peak usage:
normal: 5.0Mb
generational: 6.6Mb
normal, with "step" after each alloc: 1.0Mb
generational, with "step" after each alloc: 2.2Mb

Details (allocated memory oret runtime):

#### "step" disabled, "gen" disabled###########################################


--------------------------------------------------------------------------------
Command:            src/lua /home/juris/P/gengc-test.lua
Massif arguments:   --time-unit=i
ms_print arguments: massif.out.6062
--------------------------------------------------------------------------------


    MB
4.962^                                                            #
     |                                                           @#:
     |                                                       ::::@#::
     |                                                    :@@::: @#::
     |                                                  :::@ ::: @#::::
     |                                                :::::@ ::: @#::: @:
     |                                            :::::::::@ ::: @#::: @:
     |                                            ::: :::::@ ::: @#::: @:::
     |                      @                   :@::: :::::@ ::: @#::: @:::
     |                  ::::@::              ::::@::: :::::@ ::: @#::: @:::

| @::: : @: ::: ::@::: :::::@ ::: @#:::@::::| @@: : : @: : @@::: ::@::: :::::@ ::: @#:::@::::| ::@@: : : @: ::: :::@ ::: ::@::: :::::@ ::: @#:::@:::::| ::: @@: : : @: :: :::::@ ::: ::@::: :::::@ ::: @#:::@:::::| :::: : @@: : : @: :: ::: :::@ ::: ::@::: :::::@ ::: @#:::@:::::| :::: : : @@: : : @: :: : : :::@ ::: ::@::: :::::@ ::: @#:::@::::::| :@@:::: : : @@: : : @: :: : : :::@ ::: ::@::: :::::@ ::: @#:::@::::::| ::@ :::: : : @@: : : @: :: : : :::@ ::: ::@::: :::::@ ::: @#:::@::::::| ::@ :::: : : @@: : : @: :: : : :::@ ::: ::@::: :::::@ ::: @#:::@::::::| ::@ :::: : : @@: : : @: :: : : :::@ ::: ::@::: :::::@ ::: @#:::@::::::0+----------------------------------------------------------------------->Mi0443.1

#### "step" enabled, "gen" disabled:###########################################


--------------------------------------------------------------------------------
Command:            src/lua /home/juris/P/gengc-test.lua
Massif arguments:   --time-unit=i
ms_print arguments: massif.out.6079
--------------------------------------------------------------------------------


    KB
993.8^               :

|#:@::::::::::@::::::::::::::::::::@:::::::::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:| #:@:: : :::::@::: :::::::::::: :::@:::: :::::@::::@:::::@:::::@:::::@:0+----------------------------------------------------------------------->Mi0828.5

#### "step" disabled, "gen" enabled:###########################################


--------------------------------------------------------------------------------
Command:            src/lua /home/juris/P/gengc-test.lua
Massif arguments:   --time-unit=i
ms_print arguments: massif.out.6070
--------------------------------------------------------------------------------


    MB
6.615^                                                               #
     |                                                          :::::#
     |                                                       ::::::  #:
     |                                                       ::::::  #:
     |                                                     ::::::::  #::
     |                                                  :@@: ::::::  #::
     |                                               ::::@ : ::::::  #::
     |                                           ::::::::@ : ::::::  #:::
     |                                        :::::: ::::@ : ::::::  #:::
     |                                      :::: ::: ::::@ : ::::::  #::::
     |                                    :::::: ::: ::::@ : ::::::  #::::
     |                                  :::::::: ::: ::::@ : ::::::  #::::
     |                              :@::: :::::: ::: ::::@ : ::::::  #::::
     |             ::@           ::::@: : :::::: ::: ::::@ : ::::::  #:::::
     |           ::::@        @::::::@: : :::::: ::: ::::@ : ::::::  #:::::

| ::::::::@ @@:@::::::@: : :::::: ::: ::::@ : ::::::#:::::::@| :::: :::::@: @::@ :@::::::@: : :::::: ::: ::::@ : ::::::#:::::: @| ::: :: :::::@::@::@ :@::::::@: : :::::: ::: ::::@ : ::::::#:::::: @| @@::: :: :::::@::@::@ :@::::::@: : :::::: ::: ::::@ : ::::::#:::::: @| @ ::: :: :::::@::@::@ :@::::::@: : :::::: ::: ::::@ : ::::::#:::::: @0+----------------------------------------------------------------------->Mi0480.7

#### "step" enabled, "gen" enabled:############################################


--------------------------------------------------------------------------------
Command:            src/lua /home/juris/P/gengc-test.lua
Massif arguments:   --time-unit=i
ms_print arguments: massif.out.6089
--------------------------------------------------------------------------------


    MB
2.234^                                 #

| @@# :@ :::::| : :@ # ::@ ::::::::| @ : ::@ # :::@ :::::@::::| @@ @: :::@ # @:::@ :@::::@::::| :@@ ::@::: ::::@ # :@:::@ ::@::::::@:::::| ::@@ :: @:: :::::@ # ::@:::@: :::@:::::::@:::::| ::::@@ ::: @:: :::::@ # :::@:::@: ::::@::::::::@:::::| : ::@@ @::: @:: :::::::@ # @:::@:::@: :::::@:::::::::@:::::| ::: ::@@ :@::: @:: @:::::::@ # @:::@:::@: ::::::@::::@:::::@:::::| : : ::@@ @:@::: @:: @:::::::@#::@:::@:::@::::::::@:::::@:::::@::::::| :: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@| ::: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@| ::: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@| ::: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@| ::: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@| ::: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@| ::: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@| ::: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@| ::: : ::@@:@:@::: @:: @:::::::@ #:@:::@:::@::::::::@:::::@:::::@:::::@0+----------------------------------------------------------------------->Mi0581.1

References:
- A review of changes between 5.1 and 5.2-work3, Peter Cawley
- Re: A review of changes between 5.1 and 5.2-work3, Jerome Vuarand
- Re: A review of changes between 5.1 and 5.2-work3, Peter Cawley
- Re: A review of changes between 5.1 and 5.2-work3, Mike Pall
- Re: A review of changes between 5.1 and 5.2-work3, Roberto Ierusalimschy

Prev by Date: Re: OT complied language most like Lua?
Next by Date: Re: The removal of function environments: An opportunity for optimization?
Previous by thread: Re: A review of changes between 5.1 and 5.2-work3
Next by thread: Re: A review of changes between 5.1 and 5.2-work3
Index(es):
- Date
- Thread