• Subject: Re: bug report: string.format rounding error
• From: Thomas Jenni <Thomas.Jenni@...>
• Date: Thu, 19 Sep 2019 07:23:59 +0000

 From a decimal perspective the binary rounding seems strange.  Especially if the representation is not exact, one has to trust  the hidden "round to nearest“ function as one can see in the  following example. -- test n={6.05,6.15,6.25,6.35,6.45,6.55,6.65,6.75,6.85,6.95} for i = 1,#n do x = string.format("%.1f",n[i]) print(n[i].." "..x.." "..(x-n[i])) end -- output 6.05 6.0 -0.05 6.15 6.2 0.05 6.25 6.2 -0.05 6.35 6.3 -0.05 6.45 6.5 0.05 6.55 6.5 -0.05 6.65 6.7 0.05 6.75 6.8 0.05 6.85 6.8 -0.05 6.95 7.0 0.05 Nevertheless it is the right way to round with as less bias as possible. So i agree with you, that it's a feature and not a bug. Thx for the  patience and explanations.  Thomas Am 18.09.2019 um 23:55 schrieb Tim Hill : On Sep 18, 2019, at 9:12 PM, Coda Highland wrote: On Wed, Sep 18, 2019 at 3:06 PM Tim Hill wrote: On Sep 18, 2019, at 6:24 PM, Coda Highland wrote: On Wed, Sep 18, 2019 at 12:21 PM Coda Highland wrote: However, there are six different ways that a C implementation could choose to implement number formatting, five of which are documented in IEEE-754: * Round towards 0 * Round towards +inf * Round towards -inf * Round towards nearest, ties to even * Round towards nearest, ties away from 0 * Truncate decimal representation It occurs to me that truncating the decimal representation is actually equivalent to rounding towards 0, so that's only five after all and all of them are documented. Wasn't thinking about that. /s/ Adam Not for negative numbers. Yes for negative numbers. If you write -1.23456, and you truncate that representation (e.g. to -1.23), then you have rounded towards zero. Besides, if it WERE different for negative numbers, then it would be equivalent to rounding towards -inf instead, which would still mean it's not a distinct rounding mode. /s/ Adam Lol you’re right. What WAS I drinking last night? :) —Tim