Re: [patch] fix luai_num* null states

[Lourival Vieira Neto <lourival.neto@...>]

On Wed, Mar 5, 2014 at 2:09 PM, Roberto Ierusalimschy
<roberto@inf.puc-rio.br> wrote:
>> On Tue, Mar 4, 2014 at 3:34 PM, Roberto Ierusalimschy
>> <roberto@inf.puc-rio.br> wrote:
>> >> > I had another small issue on luaV_numtointeger(). I had an 'integer
>> >> > overflow' on the line 81 of lvm.c. I solved as follows:
>> >> >
>> >> > -  if (cast_num(MIN_INTEGER) <= n && n < (MAX_INTEGER + cast_num(1))) {
>> >> > +  if (cast_num(MIN_INTEGER) <= n && (n - cast_num(1)) < MAX_INTEGER) {
>> >>
>> >> This is tricky... In my machine, this change does not pass the standard
>> >> tests for vanilla Lua (64-bit floats and integers). (I am trying to figure
>> >> out why, but when I add some printfs to the code, it works again; weird...)
>> >
>> > With the new code, the machine does all computations for
>> > (n - cast_num(1)) < MAX_INTEGER in registers, with more precision than
>> > 64 bits. We do need the "overflow" here to get the right result.
>> > (Think when 'n' is 2.0^63, which does not fit into a 64-bit integer.)
>>
>> Sorry,  it really sounds tricky.. I don't get it yet =(.. in that
>> case, we should have a float comparison, don't we? Thus, the '<'
>> expression would return 0, the 'if' test would fail and the function
>> would return 0, which should be the right result for n = 2.0^63. Is
>> that wrong? I must be missing something.
>
> MAX_INTEGER is stored as a 64-bit float constant, so it is rounded
> to 2^63. Then, with the computation done with 80 bits, we have
> (2^63 - 1 < 2^63) done precisely, resulting in true. (In the original
> code, the whole (MAX_INTEGER + cast_num(1)) is stored as a 64-bit constant,
> correctly rounded to 2^63. The computation now is 2^63 < 2^63, which is
> false.)

Weird.. this precision difference doesn't happen in my machine.. in
both cases I've got false.. I've attached a code sample which gives me
the following output:

lneto@coruscant /tmp $ ./cmp
n:               9223372036854775808.000000
LLONG_MAX + 1.0: 9223372036854775808.000000
n - 1.0:         9223372036854775808.000000
original: false
modified: false

Regards,
-- 
Lourival Vieira Neto

#include <stdio.h>
#include <math.h>
#include <limits.h>

int main(void)
{
  double n = pow(2,63);
  
  printf("n:\t\t %f\n", n);
  printf("LLONG_MAX + 1.0: %f\n", LLONG_MAX + 1.0);
  printf("n - 1.0:\t %f\n", n - 1.0);

  if (n < (LLONG_MAX + 1.0))
    printf("original: true\n");
  else
    printf("original: false\n");

  if ((n - 1.0) < LLONG_MAX)
    printf("modified: true\n");
  else
    printf("modified: false\n");

  return 0;
}