Re: Setting Float Precision in Lua.c

[KHMan <keinhong@...>]

On 6/5/2018 8:33 AM, Albert Chan wrote:
> Some operation system, say Windows, use EXTENDED PRECISION by default.
> (64 bits long double instead of 53 bits double)
>
> Example, from my Win7 laptop:
>
> Python return the correct rounded sum:
> (thus *all* Python import modules also does 53-bits float rounding)
>
> > > > 1e16 + 2.9999           # good way to test float precision setting
> 10000000000000002.0  # Python got the rounding right
>
> Lua 5.4.0 (work1)  Copyright (C) 1994-2018 <tel:1994-2018> Lua.org 
> <http://lua.org/>, PUC-Rio
> > string.format('%.17g\n', 1e16 + 2.9999)
> 10000000000000004    -- wrong, rounding in long double setting
>
> For 53 bits double, to make same math result across platform,
> maybe add this to Lua.c ? (even if not needed)
>
> fesetenv (FE_PC53_ENV);

Wrong, Lua coders should instead strive to have a modicum level of 
competency in floating-point arithmetic and floating-point 
implementations.

If someone insist on _PERFECTION_ for their floating-point output, 
then IMHO the onus is on that someone to do (and maintain) the 
necessary tweaks (and test regime) for that purpose.

If a calculation such as "1e16 + 2.9999" is _IMPORTANT_ for a 
program, then the program should do checks to ensure that it has 
been compiled per requirements on problematic platforms.*

I really do not think suppliers of programming language 
implementations should be hand-holding or babysitting users who 
want perfection down to the very last ULP. Those users are 
cordially invited to do it on their own time and dime.


*Well, it's probably a defective program anyway in that it is very 
sensitive to digits in the ULP. More chaotic output than reliable 
output. Reminded me of the story of the aeronautical grad who 
found that his sim gave wildly different results on different chip 
platforms. This made his project results more or less unusable.

--
Cheers,
Kein-Hong Man (esq.)
Selangor, Malaysia