Re: LuaJIT performance

[Mike Pall <mikelu-0908@...>]

Mark Hamburg wrote:
>> Most of the LJ2 VM is already "64 bit ready". E.g. it has an
>> arch-independent 32 bit pointer abstraction for all GC objects.
>
> Handles? A limited range 64-bit pointer?

32 bit pointers on a 64 bit system. :-)

They just have to reside in the lowest 4 GB (actually it's more
efficient to only use the lowest 2 GB). That's easy to guarantee
since LJ2 already uses a (much faster) bundled memory allocator.
You can turn it off in 32 bit mode, but not in 64 bit mode.

The argument for restricting the VM to max. 4 GB is that this only
applies to the objects managed by the VM itself. You can still
load many gigabytes into memory, since all those data-intensive
libraries (like libpng) use their own memory management and give
you a pointer or handle which can be stored in a tiny userdata.

The other argument is that given typical object sizes you can
estimate the time for a full GC traversal by the time for random
L2 cache misses. That's probably several seconds for a GB. Or in
other words: the Lua GC currently is just not up to the task to
manage billions of tiny objects, anyway.

If someone, somehow eventually outgrows the 4G limit one could
switch to compressed pointers. Shifting the pointer by 3 or 4 bits
results in a 32 GB or 64 GB limit.

[All of that is needed to preserve the special kind of tagged
values which overlap with a double and use only 64 bits.]

--Mike