lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


 > rint(0.50001)
 0

I get 1 as the result on my version, not sure how you could get otherwise.  The addition should populate the guard and sticky bits which should then round up.

This behavior seems to be system-dependent.
On my home computer I get 1 on Windows and 0 on Linux.
(this is Lua 5.1, but FPU rounding logic should not depend on Lua version)

Bleurgh... apparently it depends on your compiler version and switches.  I've done some research and it seems that the 80 bit representation in the x87 co-processor breaks things badly.  It can cause double rounding to round up when it shouldn't (since it rounds to 80 bits and then 64 bits) or it can cause round downs when it shouldn't because GCC sometimes needs to free a x87 register and apparently truncates the result into a regular 64 bit register when it 'spills' the value (which I assume must be what is triggering the issue for you).

The most commonly proposed solution seems to be to ensure that you compile with `-msse2 -mfpmath=sse` to use the SSE unit which does IEEE compliant double precision math.

I haven't done any experimentation with these issues yet but I suspect it may even depend on your compiler version.  Apparently the double rounding issue should be triggerable with the value 262144.75, but the spilling issue will depend on exactly the code optimisation for when it runs out of registers.