Re: Crash in luaL_loadfile in 64-bit x86_64

[Mike Pall <mikelu-0910@...>]

E. Wing wrote:
> Since I needed to be able to hold 64-bit numbers, I changed
> lua_Number to long double.

Well, there's your problem ... sizeof(long double) is 12 bytes for
the x86 OSX ABI, but 16 bytes for the x64 OSX ABI (due to stricter
alignment rules).

But the stores from the x87 FPU to a long double only write to the
first 12 bytes. The remaining 4 bytes of memory on x64 still
contain random garbage from a previous use.

Alas, the function hashnum() in ltable.c reads sizeof(lua_Number)
bytes to generate a hash for a lua_Number. I.e. on 64 bit this
includes the 4 random bytes, but not on 32 bit.

This means the same numeric key could hash to different values.
This confuses the code in newkey(), because it tries to find a
colliding key in the wrong hash chain.

One solution is to avoid long double (it's pretty slow anyway) and
instead use lightuserdata to store 64 bit numbers. The other
solution is to modify ltable.c: hardcode the numints define to 3.

[Ok, where do I send the invoice? ;-)]

--Mike