[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: bug report: string.format rounding error
- From: Philippe Verdy <verdyp@...>
- Date: Wed, 18 Sep 2019 19:22:18 +0200
May be the rounding mode may explain the difference. Note you display only 2 decimals and the remainder .25 is correctly rounded to 0.
> Perhaps not, and I'm not defending the practice, but it IS an annoyingly common
> thing to ask about, especially since it's not only not a bug in Lua, it's not a bug at
> all and it's not specific to Lua. It's something that every developer needs to know.
Knowledge about binary floating point is certainly something every developer who
handles number should know about.
In this particular case, though...
1.625 is exactly representable in IEEE754 (float and double). There is no loss of
Information from the conversion. I would have expected both inputs to produce
a result of 1.63.
A quick attempt shows the following:
- Lua 5.1 (32-bit Windows)
- Lua 5.2 (32-bit Windows)
- C (Gcc 4.8.1)
- C (Visual Studio 2017)
1.63 after rounding for both literals
- Lua 5.3 (32-bit Windows)
So, yes, it is a valid question as to why there is a different result for the same input.
I don't have an explanation.