lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


Interesting question: because  you demonstrate that the number  0.30000000000000004 (with 17 significant decimal digits) is different from the number 0.3 but is still converted to the same string '0.3' by tostring().

And that's an inconsistency of Lua's implementation of tostring(number) that discards some significant precision in numbers, to only 14 significant decimal digits (excluding the most significant 0's) instead of 17 as required for this representation of Lua numbers based on IEEE 64-bit doubles (the 17th digit is needed because of rounding for the inverse operation, with numbers of any exponential scale, because the range of integers represented exactly in those doubles can have up to 16 digits but not their maximum and rounding takes effect but with 17 decimal digits the rounding will always lead to the expected binary64 number). 

And the assumption that two different Lua numbers should have two different string representations with the tostring() function is then false in this implementation: that conversion of numbers to strings is lossy, not reversible.

So tostring() performs some (unspecified?) rounding before the actual conversion, probably because it was felt "sufficient" to avoid "overlong" strings (and then save up to 3 bytes in strings)...

With this implementation, the 14 decimal digits, a possible dot, a possible negative sign, the "e" prefix for a possible sign of the decimal exponent, and up to 3 digits for an absolute decimal exponent gives a string up to 20 bytes (instead of 23), and I don't see a rationale for limiting the string length... except to "hide" the rounding errors that unavoidably occur when using trigonometric functions (like sin or cos with some some multiples of pi), or exponential functions (with ^ or log with base 10), or unexact arithmetic (like divisions by non-powers of two), which could cause problems in badly written financial/accounting applications (storing amounts directly as numbers with 2 or 3 fractional decimals, without scaling them first to integers before adding them and without proper roundings when needed).

As well, if the 17 digits were displayed and correctly rounded, the amount 0.10+0.10+0.10 would display 0.30000000000000004 with the unrounded tostring() formatter by default (and not 0.30 when using a better formatter); it would not be wrong but may not fit well in tabular columns. But even in this case tostring(0.30000000000000004) still doesn't display '0.30' but '0.3', which does not correctly line up in tabular results (where there should be a fixed amount of decimals).

So that rationale is IMHO probably broken: the default rounding to 14 decimal digits in tostring() is only a facility for lazy programmers of financial/accounting applications, or for Lua programming beginners. You'd better override that builtin function in your Lua programs/modules/libraries to replace that (stupid) default.




Le ven. 31 mai 2019 à 17:02, Robert Virding <rvirding@gmail.com> a écrit :
When does Lua decide to trim floats when doing a tostring? For example:

> "a"..tostring(0.30000000000000004).."b"
a0.3b
> "a"..tostring(0.3000000000000004).."b"
a0.3b
> "a"..tostring(0.300000000000004).."b"
a0.3b
> "a"..tostring(0.30000000000004).."b"
a0.30000000000004b

These are all valid floats. Added the a adn b to make the string clearer.

> 0.3 ~= 0.1+0.1+0.1
true
> 0.30000000000000004 == 0.1+0.1+0.1
true

I am interested as I am implementing Lua and want it to behave the same way.

Robert