Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

[Roberto Ierusalimschy <roberto@...>]

> > I get 0 on all platforms I can test Lua 5.[12] on, which is of course as bad
> > as 3 but not *this* bad. Could any patches you use have affected this?
> 
> On Mac OS X 10.11.6 I get
> 
> Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
> > print(1e97 % 97)
> 0
> 
> Lua 5.2.4  Copyright (C) 1994-2015 Lua.org, PUC-Rio
> > print(1e97 % 97)
> 0
> 
> Lua 5.3.4  Copyright (C) 1994-2017 Lua.org, PUC-Rio
> > print(1e97 % 97)
> 3.0

Lua 5.1/5.2 computed the module (a%b) with (a - floor(a/b) * b),
which may introduce other rouding errors. Lua 5.3 computes the module
with 'fmod'; at least in my machine, the only error is the one in the
representation of 1e97 itself (see my other messages).

-- Roberto