Re: Unexpected calculation result with Lua53

[Enrico Colombini <erix@...>]

On 02-May-15 22:13, tonyp@acm.org wrote:
> function fact(n)
>    if n == 0 then return 1 end
>    return fact(n-1) * n
> end
>
> print(fact(66),fact(66.))
>
> Using fp parameter, return correct (?) result
> Using integer parameter, return 0 (zero) result!!!!

This can be simplified to:

> -9223372036854775808 * 65
-9223372036854775808

> -9223372036854775808 * 66
0

> -9223372036854775808 * 67
-9223372036854775808

> -9223372036854775808 * 68
0

I am by no means expert of the new int/fp dualism, so I do not try to 
explain the above alternance, but I note that the integer maxed out:

> string.format("%x", -9223372036854775808)
8000000000000000

You can also get signed integer wrap:

> fact(60)
-8718968878589280256

The last result that can be contained in a 64-bit integer seems to be 
fact(20):

> fact(20.0) < 2^64
true

> fact(21.0) < 2^64
false

--
  Enrico