Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

[Dirk Laurie <dirk.laurie@...>]

2018-06-13 2:13 GMT+02:00 Albert Chan <albertmcchan@yahoo.com>:
> Thanks Hartmut for the HP-15C test.
> It seems HP-15C does *decimal* math, 10 significant digits, half-round-up
>
> s = (a + b + c) / 2
> = (1e5 + 99999.99979 + 0.00029) / 2
> = (199999.9998 + 0.00029) / 2   -- round up
> = 200000.0001 / 2                         -- round up again
> = 100000.0001                               -- half-round-up
>
> I have never seen a calculator like this ...

I have an HP41C, but it no longer works. I can recall, however, that
it correctly evaluated 1e97 mod 97. Default Lua 5.3 on my machine says
"1e97"%"97" is 3, which is wrong. Lua 5.3 built as `make linux -e
MYCFLAGS=-DLUA_NOCVTS2N` and with a module supplying appropriate
string metamethods, gets it right. My much more modern calculator of a
different brand also gets it wrong.)