Re: [5.3] Converting a floating point number to integer.

[Jan Behrens <jbe-lua-l@...>]

On Wed, 19 Nov 2014 11:03:38 -0200
Roberto Ierusalimschy <roberto@inf.puc-rio.br> wrote:

> > I have a question:
> > 
> > Is there any problem defining x//y := math.floor(x/y) for all cases
> > where either x or y are not integral?
> 
> No, just it would be a different operations. (Currently, // is integer
> division, not a division with integer results.)
> 
> Consider the following examples (assuming your proposal):
> 
>   x = (1 << 60) + 2
>   print(x // 2)     --> 576460752303423489
>   print(x // 2.0)   --> 576460752303423488
> 
>   x = (1 << 62)
>   print(x // 3)     --> 1537228672809129301
>   print(x // 3.0)   --> 1537228672809129216
> 
> In both cases, the result is an integer, there was no overflow at the
> computation, but the results are different.

How about modifying the idea as follows:

1. If both x and y are of integer subtype,
   then x//y is integer division.

2. If both x and y can be represented as an integer, then
   x//y := math.tointger(x)//math.tointeger(y)

3. In all other cases, x//y := math.floor(x/y)

That way

x = (1 << 62)
print(x // 3)     --> 1537228672809129301
print(x // 3.0)   --> 1537228672809129301


> A key point in Lua is that
> any operation performed on integers or floats give exactly the same
> result, unless either an operand or the result cannot be expressed in
> the given representation (integer or float).

The modified proposal above does fulfil that objective, doesn't it?


> But that is not a big problem. We could as well define it to be x//y
> := math.floor(x/y)+0.0 (that is, giving a float as result for float
> operands). It is a matter of choice.
> 
> -- Roberto

That would be another alternative. However, math.floor() already
returns a subtype depending on the argument's value range.

Consider the following expressions in Lua 5.3.0-beta:

    f = 2.0^62.0  -- integral float within integer range (2^-63..2^63-1)
    g = 2.0^63.0  -- integral float exceeding integer range

    i = math.floor(f)
    j = math.floor(g)

    print(i*i)  --> 0
    print(j*j)  --> 8.5070591730235e+37

    -- Why does this happen? Because:

    print(i)  --> 4611686018427387904  -- integer
    print(j)  --> 9.2233720368548e+18  -- float

Therefore math.floor()'s subtype behavior is not just dependent on the
subtype of its argument, but dependent on its particular value. The
"//" operator could behave in a similar fashion. 


Regards
Jan