[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: why tostring for number use LUA_NUMBER_FMT "%.14g", not "%.16g" ?
- From: Coda Highland <chighland@...>
- Date: Sun, 8 Sep 2013 00:09:37 -0700
On Sat, Sep 7, 2013 at 1:51 PM, Leo Razoumov <email@example.com> wrote:
> On 9/7/13, pulleyzzz_gmail <firstname.lastname@example.org> wrote:
>> for this code:
>> print(1/3==tonumber(a)) -- false
>> --if use .16g
>> print(1/3==tonumber(a)) -- true
> Changing format to "%.16g" would not solve your problem of comparing
> floats with == operator.
> 1/3 is computed in Lua natively in binary representation.
> a=string.format('%.16g',1/3) is a binary representation converted to a
> finite length
> base-10 number. Very often it cannot be done without loss of accuracy.
> Next, you are converting it to binary again with tonumber(a) which
> causes another loss of accuracy.
> As a rule of thumb do not compare floats with == operator. It will
> lead to subtle and
> difficult to catch errors.
This answer, while technically correct, is actually misguided.
1/3 == 1/3 is guaranteed to be true. It's the same expression. (1/6 +
1/6 == 1/3 is a different question.)
But when you're discussing SERIALIZING floats, %.14g is not
sufficient. %.16g is sufficient to uniquely identify every possible
double-precision IEEE floating point number with a clean round trip.
So there IS a valid question to be asked here: Why SHOULDN'T
tostring() use %.16g?