Re: Floating point

[Paul K <paul@...>]

Hi Iain,

> I don’t understand why I can’t print the floating point value of this variable. I haven’t rounded or truncated it, so why does it print exactly 116, even though that’s not its value?

It's because the default format only uses 14 digits (LUA_NUMBER_FMT
"%.14g") and it's not enough to represent the number.

Try ("%.17g"):format(23.2/0.2); this prints: 115.99999999999999, which
should explain the results you see (some more details on this:
http://stackoverflow.com/a/13540020/1442917).

Paul.

On Thu, Oct 29, 2015 at 10:16 AM, iain morland <iain@iainmorland.net> wrote:
> Hi all,
>
> If I assign a variable to the result of 23.2/0.2, and print that variable, Lua outputs 116.
>
> However if I then use the same variable as the limit in an iterator, the iterator stops on the 115th repeat, rather than the 116th as expected.
>
> So it therefore seems that the variable is a floating point number between 115 and 116.
>
> I don’t understand why I can’t print the floating point value of this variable. I haven’t rounded or truncated it, so why does it print exactly 116, even though that’s not its value?
>
> TIA,
> Iain
>
>