lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


> That's arbitrary and not a solution in any sense of the word.

You are right, using 14 digits to display a number, from a 53 bits
mantissa floating point, is arbitrary.
But it comes from a compromise between the precision we would like to
show and the precision we can be assure.
My suggestion is less arbitrary: it is aimed to merge the behavior of
tostring() result with the underneath value:

a=1.234567890123456
b=1.234567890123499
print(a, b)     --> 1.2345678901235 1.2345678901235
print(tostring(a)==tostring(b)) --> true
print(a==b) --> false, but with my proposition: true

The rationale is: because a and b are two close to be differentiated
when converting to a string, why would they be treated as different in
the number context?
Or else: if they are considered different as numbers, why do we accept
that their string representation is equal?
I know this is not a real "solution", but it would be a step toward
avoiding a lot of rounding problems with floating point numbers.