[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)
- From: Roberto Ierusalimschy <roberto@...>
- Date: Wed, 27 Jun 2018 10:47:09 -0300
> As already explained, what is wrong in Lua 5.3 is 1e97. Floating-point
> rounding...
>
> Correct 1e97:
> 0x4ae825771dc07672ddd0f8e8ac39250971ac4210cecb6f656caeb910a000000000000000000000000
>
> Representation of 1e97 as an IEEE double:
> 0x4ae825771dc0780000000000000000000000000000000000000000000000000000000000000000000
Just to be sure:
0x4ae825771dc07672ddd0f8e8ac39250971ac4210cecb6f656caeb910a000000000000000000000000 % 97 is 10
0x4ae825771dc0780000000000000000000000000000000000000000000000000000000000000000000 % 97 is 3
-- Roberto
