Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

[Tim Hill <drtimhill@...>]

> On Jun 26, 2018, at 2:52 AM, Dirk Laurie <dirk.laurie@gmail.com> wrote:
> 
> 2018-06-13 2:13 GMT+02:00 Albert Chan <albertmcchan@yahoo.com>:
>> Thanks Hartmut for the HP-15C test.
>> It seems HP-15C does *decimal* math, 10 significant digits, half-round-up
>> 
>> s = (a + b + c) / 2
>> = (1e5 + 99999.99979 + 0.00029) / 2
>> = (199999.9998 + 0.00029) / 2   -- round up
>> = 200000.0001 / 2                         -- round up again
>> = 100000.0001                               -- half-round-up
>> 
>> I have never seen a calculator like this ...
> 
> I have an HP41C, but it no longer works. I can recall, however, that
> it correctly evaluated 1e97 mod 97. Default Lua 5.3 on my machine says
> "1e97"%"97" is 3, which is wrong. Lua 5.3 built as `make linux -e
> MYCFLAGS=-DLUA_NOCVTS2N` and with a module supplying appropriate
> string metamethods, gets it right. My much more modern calculator of a
> different brand also gets it wrong.)
> 

My HP41C still works (those were the days, HP), and yes you are correct.

—Tim