Re: Do LuaJit 2 support more than 2GB memory allocation on FreeBSD/x64 now?

[Miles Bader <miles@...>]

Mike Pall <mikelu-1204@mike.de> writes:
>> I just want to make sure that now LuaJit 2 whether surport more
>> than 2GB memory allocation on FreeBSD/x64 or not.
>
> Rest assured, you do NOT want to allocate more than that amount of
> Lua objects. The garbage collector in LuaJIT 2.0 (which is the
> same as Lua 5.1) is simply not up to the task. Your program would
> most certainly slow to a crawl with that many live objects.

Hmm, I've exceeded LuaJIT's memory limits, and haven't noticed any
_obvious_ slowness as I approached the limit...

Much of the memory use is in the form of humongous Lua strings (which
then I pass as input to LPEG), but there's plenty of other stuff
as... I don't think the strings count for more than about 1GB or so.
[I'm guessing strings don't count so much because they're single large
objects?]

Using Lua 5.1.4, it hit about 2.7GB of RSS, with about 1GB or less of
that due to the big strings.  By changing the way LPEG is invoked
(from "parse the whole thing" to "parse small parts of the big
string"), to try and prevent LPEG from holding onto so much, the
memory usage goes down to about 1.4GB (and so LuaJIT becomes usable),
so I'm guessing the difference between 2.7GB and 1.4GB is due to lots
of small LPEG captures.]

I dunno, maybe with better GC algos it would be twice as fast... but
certainly I've found it to be very usable as it is.

-miles

-- 
P.S.  All information contained in the above letter is false,
      for reasons of military security.