[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
**Subject**: **Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)**
**From**: Albert Chan <albertmcchan@...>
**Date**: Tue, 26 Jun 2018 12:37:02 -0400

> On Jun 26, 2018, at 5:52 AM, Dirk Laurie <dirk.laurie@gmail.com> wrote:
>
> I have an HP41C, but it no longer works. I can recall, however, that
> it correctly evaluated 1e97 mod 97. Default Lua 5.3 on my machine says
> "1e97"%"97" is 3, which is wrong.
HP41C does BCD math, "1e97" converted to 10^97 (nice ...)
HP41C must have a dedicated MOD key.
Otherwise, x % y == x - int(x / y) * y will likely loses all precision.
Lua convert "1e97" to hexfloat = 0x1.2ba095dc7701ep+322
Since Lua can only see the hexfloat, 3 is also right (w/ hexfloat)
Wrong result is due to strtod error, trying to fit "1e97" to 53 bits.
To confirm: I use my rpn: https://github.com/achan001/MAPM-5
> rpn
1e97 g 97 % ? # Lua way, g = exact hexfloat
3
1e97 97 % ? # HP41C way
10
I just downloaded WP31s *FREE* app to my ipad.
It does BCD math (16 decimal digits, halfway round-to-even)
WP31s: 1e97 ENTER 97 f-MORE MOD ==> 10
Since 97 is prime, 10 ^ 97 % 97 = 10 (Fermat little theorem)
--> HP41C and WP31s both get the mod right.