Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

[Albert Chan <albertmcchan@...>]

> On Jun 26, 2018, at 5:52 AM, Dirk Laurie <dirk.laurie@gmail.com> wrote:
> 
> I have an HP41C, but it no longer works. I can recall, however, that
> it correctly evaluated 1e97 mod 97. Default Lua 5.3 on my machine says
> "1e97"%"97" is 3, which is wrong.

HP41C does BCD math, "1e97" converted to 10^97 (nice ...)

HP41C must have a dedicated MOD key.
Otherwise, x % y == x - int(x / y) * y will likely loses all precision.

Lua convert "1e97" to hexfloat = 0x1.2ba095dc7701ep+322
Since Lua can only see the hexfloat, 3 is also right (w/ hexfloat)
Wrong result is due to strtod error, trying to fit "1e97" to 53 bits.

To confirm: I use my rpn: https://github.com/achan001/MAPM-5

> rpn
1e97 g 97 % ?     # Lua way, g = exact hexfloat
3

1e97 97 % ?        # HP41C way
10

I just downloaded WP31s *FREE* app to my ipad.
It does BCD math (16 decimal digits, halfway round-to-even)

WP31s: 1e97 ENTER 97 f-MORE MOD ==> 10

Since 97 is prime, 10 ^ 97 % 97 = 10 (Fermat little theorem)
--> HP41C and WP31s both get the mod right.