Re: Setting Float Precision in Lua.c

[Albert Chan <albertmcchan@...>]

> Why don't you compile your binaries for SSE2 only? Even easier, just compile to 64-bit binaries? Surprising you mentioned Windows uses extended precision by default when there is x64 on every 64-bit capable Intel/AMD/other chip... and has been so for many, many years already.

I already picked 53-bits roundings.
I use my own laptop behavior just as a example, same with fsum.lua

David Gay's dtoa.c strtod maybe a better example.
With 53-bits roundings, it optimized away common cases [1]

strtod("123456789e-20", NULL)
= 123456789 / 1e20   -- both numbers exactly represented in double
= 1.23456789e-012    -- division guaranteed correct rounding

> Avoid the 8087 datapath, it's not hard to avoid that legacy thing.

seems we are in agreement here.

But, if you believe in this, why not add it to Lua ?
This will automatically let everyone avoid it too.

> It's a banana skin, just walk around the banana skin. The rest of us, we don't need to be walking there.

You already are walking there, just dont know it.
It is hidden behind Lua default of showing 8 significant digits.

Python3 default to show just enough digits to round-trip [2]
So, even 1 ULP difference will shows up in the output.

Should Lua put the banana skin in the trash ?

--

[1] https://www.exploringbinary.com/fast-path-decimal-to-floating-point-conversion/
[2] https://www.exploringbinary.com/the-shortest-decimal-string-that-round-trips-examples/