>> print(string.format("%.2f",1.625000000000001))
>> 1.63
>>
>> print(string.format("%.2f",1.625000000000000))
>> 1.62
> Perhaps not, and I'm not defending the practice, but it IS an annoyingly common
> thing to ask about, especially since it's not only not a bug in Lua, it's not a bug at
> all and it's not specific to Lua. It's something that every developer needs to know.
Knowledge about binary floating point is certainly something every developer who
handles number should know about.
In this particular case, though...
1.625 is exactly representable in IEEE754 (float and double). There is no loss of
Information from the conversion. I would have expected both inputs to produce
a result of 1.63.
A quick attempt shows the following:
- Lua 5.1 (32-bit Windows)
- Lua 5.2 (32-bit Windows)
- C (Gcc 4.8.1)
- C (Visual Studio 2017)
1.63 after rounding for both literals
- Lua 5.3 (32-bit Windows)
1.63/1.62
So, yes, it is a valid question as to why there is a different result for the same input.
I don't have an explanation.
string.format offers no particular promises about rounding behavior. Neither does the sprintf() function underlying string.format. Therefore, the particular details of this behavior aren't well-defined.
However, there are six different ways that a C implementation could choose to implement number formatting, five of which are documented in IEEE-754:
* Round towards 0
* Round towards +inf
* Round towards -inf
* Round towards nearest, ties to even
* Round towards nearest, ties away from 0
* Truncate decimal representation
The fifth one is the behavior typically taught to elementary school students and what appears to be the expected behavior in this discussion. The sixth one is the way some printf implementations have historically chosen to do it, but its behavior on values like .999... have made this rather unpopular.
However, the behavior we're seeing here is actually the fourth one, which is IEEE-754's recommended rounding behavior for binary floating-point numbers: 1.625 is a tie between 1.62 and 1.63, so it rounds to the one that ends with an even digit. On the other hand, 1.625000000000001 isn't a tie, so both round-to-nearest behaviors agree that 1.63 is the correct answer.
There was discussion in the lead-up to Lua 5.3 about whether or not control over the floating-point rounding mode should be exposed to scripts, but in the end it was not.
(As a side note: I don't know if it impacts this specific example, but just saying "32-bit Windows" isn't enough to describe the platform when talking about floating-point rounding. There's also a difference in rounding behavior depending on whether it's compiled using 53-bit-mantissa doubles or 64-bit-mantissa doubles.)
/s/ Adam
