[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)
- From: Tim Hill <drtimhill@...>
- Date: Tue, 26 Jun 2018 17:16:51 -0700
> On Jun 26, 2018, at 2:52 AM, Dirk Laurie <email@example.com> wrote:
> 2018-06-13 2:13 GMT+02:00 Albert Chan <firstname.lastname@example.org>:
>> Thanks Hartmut for the HP-15C test.
>> It seems HP-15C does *decimal* math, 10 significant digits, half-round-up
>> s = (a + b + c) / 2
>> = (1e5 + 99999.99979 + 0.00029) / 2
>> = (199999.9998 + 0.00029) / 2 -- round up
>> = 200000.0001 / 2 -- round up again
>> = 100000.0001 -- half-round-up
>> I have never seen a calculator like this ...
> I have an HP41C, but it no longer works. I can recall, however, that
> it correctly evaluated 1e97 mod 97. Default Lua 5.3 on my machine says
> "1e97"%"97" is 3, which is wrong. Lua 5.3 built as `make linux -e
> MYCFLAGS=-DLUA_NOCVTS2N` and with a module supplying appropriate
> string metamethods, gets it right. My much more modern calculator of a
> different brand also gets it wrong.)
My HP41C still works (those were the days, HP), and yes you are correct.