[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
**Subject**: **Re: Incorrect expression calculation between programming languages**
**From**: Roberto Ierusalimschy <roberto@...>
**Date**: Fri, 6 Aug 2021 10:37:16 -0300

> > This expression shows that we have several disagreements between
> > programming languages about what is the end result:
>
> Is this another instance of Rump's example?
> https://www.johndcook.com/blog/2019/11/12/rump-floating-point/
>
> BTW, the correct value seems to be 8.0349...E+12
It does not seem to be a problem only of precision. Compiling Lua with
long double still gives a 11... result: 11665910614443.38757.
The problem seems to be the '%' over negative values, which has
different semantics in different languages. The numerator for the
"%2663" operation is negative. Changing 2663 to -2663 puts Lua's
result in the 80... category: 8036090802426.1 with regular doubles,
8036090802426.099574 with long doubles.
The difference between the 80... x 82... categories seems a precision
problem, though.
-- Roberto