[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)
- From: Roberto Ierusalimschy <roberto@...>
- Date: Wed, 27 Jun 2018 10:10:41 -0300
> [spc]lucy:~>lua-51
> Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio
> > print(1e97 % 97)
> -8.8928324534259e+79
> >
> [spc]lucy:~>lua-52
> Lua 5.2.4 Copyright (C) 1994-2015 Lua.org, PUC-Rio
> > print(1e97 % 97)
> -8.8928324534259e+79
> >
> [spc]lucy:~>lua-53
> Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio
> > print(1e97 % 97)
> 3.0
> >
>
> I don't think any of these are right.
As already explained, what is wrong in Lua 5.3 is 1e97. Floating-point
rounding...
Correct 1e97:
0x4ae825771dc07672ddd0f8e8ac39250971ac4210cecb6f656caeb910a000000000000000000000000
Representation of 1e97 as an IEEE double:
0x4ae825771dc0780000000000000000000000000000000000000000000000000000000000000000000
-- Roberto
