Re: LuaJIT strange memory limit

[Arseny Vakhrushev <arseny.vakhrushev@...>]

Ok, I looked through lj_alloc.c and realized that there is probably no way for LuaJIT to use more than 1Gb in 64-bit Linux... Man, that is so sad. :-( The dream shattered.:-)

On 10.11.2010, at 17:56, Arseny Vakhrushev wrote:

> Hello, Mike and everyone!
> 
> I have met some kind of a memory limit issue while using LuaJIT. I am sorry if it has already been discussed because I haven't been following the list for a while. However, I didn't find anything related except for some FreeBSD x64 memory issues.
> 
> So, my setup is 64-bit Linux, latest git LuaJIT build out of the box, latest Lua.
> 
> test.lua
> ---------------------
> local s = {}
> 
> for i = 1, 200 do
>    local j = math.random(1, 1000000)
>    local t = s[j]
>    if not t then
>        t = {}
>        s[j] = t
>    end
>    for k = 1, 1000000 do
>        table.insert(t, [[45623452345234523452345234523452345396804958604986570954867098456097804958670
>        563470658430956830948609348507690586-0386094867-23980785-0923095860945634-059609586-34
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457
>        9456093486-09865-094365-0956-08u6980-485798563782354671238947215639876t573450972398457]])
>    end
>    print(collectgarbage 'count')
> end
> ---------------------
> 
> 
> LuaJIT
> ---------------------
>> luajit-2.0.0-beta5 test.lua 
> 8222.0166015625
> 16414.055664062
> 24606.141601562
> 32798.180664062
> 40990.313476562
> ...
> 983078.68066406
> 991270.71972656
> 999462.75878906
> 1007654.7978516
> 1015846.8369141
> 1024038.8759766
> 1032230.9150391
> PANIC: unprotected error in call to Lua API (not enough memory)
> ---------------------
> 
> 
> Lua 5.1.4
> ---------------------
>> lua test.lua 
> 16414.825195312
> 32797.431640625
> 49181.416015625
> 65565.517578125
> 81949.775390625
> ...
> 3113014.8691406
> 3129398.9707031
> 3145783.0722656
> 3162167.1738281
> 3178551.2753906
> 3194935.3769531
> 3211319.4785156
> 3227703.5800781
> 3244087.6816406
> 3260471.7832031
> 3276855.8847656
> ---------------------
> 
> 
> Furthermore, I thought about turning off the built-in memory allocator in LuaJIT, but it's mandatory for 64-bit. I ran a similar test using my production platform which is in general a multithreaded and multiprocess LuaJIT environment, and it looks like multiple threads (LuaJIT states) share one global limit (that might be somehow connected with the built-in allocator), i.e. the error comes equally faster, although I expected the states to be totally independent from each other.
> 
> Thanks for any help in advance!
> 
> // Seny