Re: Setting Float Precision in Lua.c

[Albert Chan <albertmcchan@...>]

> Double rounding cause the value to be off by +/- 1 ULP
> 
> It is OK if Lua did this consistently throughout its distribution.
> Bad rounding is compensated by higher internal float precision.
> 
> -- Albert 6/5/18

Turns out, above statement is *wrong* ... Sorry

Lua does not benefit from higher internal float precision.
To show this, try x = 1e16; y = 2.9999; z = (x + y + y + y)

(x + y + y + y) as a whole is not done in extended precision
--> Lua get hit with 3 double roundings, set z = 1e16 + 12

If above is true, the case for 53-bits rounding is even stronger.

Doing the same in C, it does it all in extended precision,
calculated the *best* double, z = 1e16 + 8