helping Was: bug report: string.format rounding error

[Andrew Starks <andrew@...>]

[off topic for Lua]

This discussion reminded me of a conversation on the Ezra Klein Show with XKCD author Randall Munroe. 

https://player.fm/series/the-ezra-klein-show/randall-munroe-the-genius-behind-xkcd

He talks at length about how it’s best to assume that a) people are really smart and not lazy and b) people don’t have the context for the problem their trying to solve. The mode in their head is leading them astray because of some flaw in their thinking early on. 

Anyway, I’d like to be more like Mr. Munroe after listening to this podcast. 

- Andrew Starks

On Sep 18, 2019, at 06:21, Meepen <meepdarknessmeep@gmail.com> wrote:

"Hey why does this happen?"

*Hands 44 page manual to other person without explaining anything*

On Wed, Sep 18, 2019, 3:36 AM <markjynx@cock.li> wrote:

On 2019-09-18 09:47, Thomas Jenni wrote:
> In Lua 5.3.5 on a 64-bit machine (macOS 10.14.6), the following two
> lines produce different results
>
> print(string.format("%.2f",1.625000000000001))
> 1.63
>
> print(string.format("%.2f",1.625000000000000))
> 1.62
>
> It seems unclear to me, why in the latter case 1.625 is rounded to
> 1.62.
> Thanks for fixing this bug.
>
> best regards
>
> T. Jenni

https://www.itu.dk/~sestoft/bachelor/IEEE754_article.pdf