Re: Unexpected calculation result with Lua53

[Roberto Ierusalimschy <roberto@...>]

> I noticed a problem (which I suspect is related to the introduction of integers in Lua53), and I’d like to know if this is ‘official’ behavior, or some sort of bug.  The behavior can be seen in this small example:
> 
> function fact(n)
>   if n == 0 then return 1 end
>   return fact(n-1) * n
> end
> 
> print(fact(66),fact(66.))
> 
> Using fp parameter, return correct (?) result
> Using integer parameter, return 0 (zero) result!!!!
> 
> I don’t know what the root of this problem but if it happens to be related to integer overflow, should it be converted to floating point, and continue ‘crunching’ rather than give a completely wrong result?

Integer arithmetic uses "wrap around" for overflows, so the 0 result.

Convert an overflow result to floating point seems tempting, but
we do not see it as a useful/practical option. First, it is quite
expensive to check multiplication overflows in ANSI C. Second, if you
are computing with integers, you probably need a correct result, not a
rough aproximation.

Raising an error would be a better option than converting to float, but
again it is expensive; the wrap around behavior, besides being cheap,
has practical uses (e.g., unsigned arithmetic).

-- Roberto