Re: Setting Float Precision in Lua.c

[KHMan <keinhong@...>]

On 6/6/2018 1:15 AM, Albert Chan wrote:
> > If someone insists on _PERFECTION_ for their floating-point output,
> > they should use the hexadecimal format. Lua has full support for
> > hexadecimal floating-point numerals.
> >
> > -- Roberto
>
> Hexadecimal format won't help

Yeah, he seems to have misunderstood me there.

> It is not the output is off, it is the float internal bits.
> Double rounding cause the value to be off by +/- 1 ULP
>
> It is OK if Lua did this consistently throughout its distribution.
> Bad rounding is compensated by higher internal float precision.
>
> I preferred rounding in 53-bits. 64-bits rounding had issues.
> https://www.vinc17.net/research/extended.en.html

Why don't you compile your binaries for SSE2 only? Even easier, 
just compile to 64-bit binaries? Surprising you mentioned Windows 
uses extended precision by default when there is x64 on every 
64-bit capable Intel/AMD/other chip... and has been so for many, 
many years already.

Avoid the 8087 datapath, it's not hard to avoid that legacy thing. 
Then everything will look like a modern 64-bit FPU datapath like 
all the other chip archs, more or less, and let's hope the ULPs 
are consistent between chip archs and the chips too, ha ha.

It's a banana skin, just walk around the banana skin. The rest of 
us, we don't need to be walking there.

--
Cheers,
Kein-Hong Man (esq.)
Selangor, Malaysia