Re: Compiler warnings when dealing with huge integers

[Roberto Ierusalimschy <roberto@...>]

> On 16-07-06 07:36 AM, Egor Skriptunoff wrote:
> > More interesting observation on Lua 5.3.3:
> > ; math.floor((-a)/(a-1))
> > -1
> > ; (-a)//(a-1)
> > -2.0
> > 
> > It was surprising for me to see these expressions have different results.
> 
> Cannot reproduce:
> 
>     Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio
>     > local a = 2^53-1 print(math.floor((-a)/(a-1)), (-a) // (a - 1))
>     -2	-2.0

The intermediate value ((-a)/(a-1)) follows different paths in each
computation, and therefore can be rounded differently.

Note that a/(a-1) is roughtly (a+1)/a (even extended precision cannot
distinguish both values), which is 1 + 1/2^53. So, we are discussing
the first bit *after* the regular double precision. If this bit is
rounded down, the value becomes 1. If it is rounded up, it is slightly
larger than 1 (and its ceil becomes 2).

-- Roberto