Re: Setting Float Precision in Lua.c

[Albert Chan <albertmcchan@...>]

>> On Jun 4, 2018, at 9:40 PM, KHMan <keinhong@gmail.com> wrote:
>> 
>> On 6/5/2018 8:33 AM, Albert Chan wrote:
>> 
>> For 53 bits double, to make same math result across platform,
>> maybe add this to Lua.c ? (even if not needed)
>> fesetenv (FE_PC53_ENV);
> 
> Wrong, Lua coders should instead strive to have a modicum level of competency in floating-point arithmetic and floating-point implementations.
> 
> If someone insist on _PERFECTION_ for their floating-point output, then IMHO the onus is on that someone to do (and maintain) the necessary tweaks (and test regime) for that purpose.
> 
> If a calculation such as "1e16 + 2.9999" is _IMPORTANT_ for a program, then the program should do checks to ensure that it has been compiled per requirements on problematic platforms.*

The problem is Lua codes cannot access float precision setting.
It had to be done in C side.

Above example showed the problem of *double rounding*
Calculation rounded (up) in 64-bits, then again (up) to 53-bits double.

What is the reason for using machine "default" setting ?
This will produce inconsistent results across platform.

> I really do not think suppliers of programming language implementations should be hand-holding or babysitting users who want perfection down to the very last ULP. Those users are cordially invited to do it on their own time and dime.
> 
> *Well, it's probably a defective program anyway in that it is very sensitive to digits in the ULP. More chaotic output than reliable output. Reminded me of the story of the aeronautical grad who found that his sim gave wildly different results on different chip platforms. This made his project results more or less

Some algorithm *required* getting the last ULP right.

Example: https://github.com/achan001/fsum

fsum.lua use Shewchuk Algorithm to get *exact* sum of numbers.
If ask for total, it returned the correctly rounded 53-bit double.
It simply will not work with extended precision rounding.

I had to patch Lua.c to make it work.