Re: GC problem with lot of allocated memory

I'm not sure

rodrigo@rodrigo-Vostro-5470:~$ cat teste5.lua
N = 1.0e7
C = {}
for i=1,N do C[i] = i end

print(collectgarbage("count"))

for i=1,10*N do
    local a = {1,2,3,4}
end

print(collectgarbage("count"))
rodrigo@rodrigo-Vostro-5470:~$ lua5.2 teste5.lua
262167.40917969    419
445742.6796875    696
rodrigo@rodrigo-Vostro-5470:~$ nano teste5.lua
rodrigo@rodrigo-Vostro-5470:~$ lua5.2 teste5.lua
262166.66113281    677
445487.28613281    293
rodrigo@rodrigo-Vostro-5470:~$ cat teste5.lua
collectgarbage("generational")
N = 1.0e7
C = {}
for i=1,N do C[i] = i end

print(collectgarbage("count"))

for i=1,10*N do
    local a = {1,2,3,4}
end

print(collectgarbage("count"))
rodrigo@rodrigo-Vostro-5470:~$

2016-05-29 22:30 GMT-03:00 Daurnimator <quae@daurnimator.com>:

This doesn't need to be something manually specified.
You haven't mentioned which version of lua use are using.
Lua 5.2 had an optional generational garbage collector, which does what you describe.
Try running your code with it on. From lua: collectgarbage("generational")

However, it was removed in 5.3 due to lack of positive results (well, as far as I understood it, lack of *any* useful reports)

On 30 May 2016 at 09:57, Rodrigo Azevedo <rodrigoams@gmail.com> wrote:
maybe an orthogonal idea:

setmetatable({},__mode='l')

where 'l' means 'long lived'

Then, each object into this table (keys and values) will be scanned by GC "almost never". This can alleviate the work of GC, and keep the memory usage more straight.

Is this possible?

2016-05-29 4:39 GMT-03:00 zejian ju <juzejian@gmail.com>:
You can try luarc (https://github.com/zenkj/luarc-5.1), Refcount GC is added upon the normal incremental Mark&Sweep GC.

The result:
> time ./lua test.lua
262172.40332031
464729.12695312

real 13m44.993s
user 13m38.374s
sys 0m0.747s

> time ./luarc test.lua
262184.49316406
262184.49316406

real 4m36.861s
user 4m37.461s
sys 0m0.137s

2016-05-29 1:03 GMT+08:00 Rodrigo Azevedo <rodrigoams@gmail.com>:
The code

N = 1.0e7
C = {}
for i=1,N do C[i] = i end
print(collectgarbage("count"))
for i=1,100*N do
local a = {1,2,3,4}
end
print(collectgarbage("count"))

prints

262166.11035156
445760.78710938

problem: when Lua uses lot of memory, the creation of temporary objects can consume all your memory. I think CG should collect the objects fast enough, but it does not. A perfectly OK program can't run because of the amount of garbage to be collected consumes all memory.

I overcome this situation using collectgarbage() options inside the "for", but the performance is severely degraded. (due to the high number of objects GC need to manager, or the manual increase of the number of cycles.)

question: There is some place of Lua source code that I can modify to keep the memory usage as close as possible to 262166.11035156 without doing my program 3x slowly?

--
Rodrigo Azevedo Moreira da Silva

--
Rodrigo Azevedo Moreira da Silva

Rodrigo Azevedo Moreira da Silva