Re: bug report: string.format rounding error

[Meepen <meepdarknessmeep@...>]

"Hey why does this happen?"

*Hands 44 page manual to other person without explaining anything*

On Wed, Sep 18, 2019, 3:36 AM <markjynx@cock.li> wrote:

On 2019-09-18 09:47, Thomas Jenni wrote:
> In Lua 5.3.5 on a 64-bit machine (macOS 10.14.6), the following two
> lines produce different results
>
> print(string.format("%.2f",1.625000000000001))
> 1.63
>
> print(string.format("%.2f",1.625000000000000))
> 1.62
>
> It seems unclear to me, why in the latter case 1.625 is rounded to
> 1.62.
> Thanks for fixing this bug.
>
> best regards
>
> T. Jenni

https://www.itu.dk/~sestoft/bachelor/IEEE754_article.pdf