Re: why tostring for number use LUA_NUMBER_FMT "%.14g", not "%.16g" ?

[Leo Razoumov <slonik.az@...>]

On 9/8/13, Coda Highland <chighland@gmail.com> wrote:
> On Sat, Sep 7, 2013 at 1:51 PM, Leo Razoumov <slonik.az@gmail.com> wrote:
>> On 9/7/13, pulleyzzz_gmail <pulleyzzz@gmail.com> wrote:
>>> for this code:
>>>
>>> a=tostring(1/3)
>>> print(1/3==tonumber(a))  -- false
>>>
>>> --if use .16g
>>>
>>> a=string.format('%.16g',1/3)
>>> print(1/3==tonumber(a))  -- true
>>>
>>
>> Changing format to "%.16g" would not solve your problem of comparing
>> floats with == operator.
>>
>> 1/3 is computed in Lua natively in binary representation.
>>
>> a=string.format('%.16g',1/3) is a binary representation converted to a
>> finite length
>> base-10 number. Very often it cannot be done without loss of accuracy.
>> Next, you are converting it to binary again with tonumber(a) which
>> causes another loss of accuracy.
>>
>> As a rule of thumb do not compare floats with == operator. It will
>> lead to subtle and
>> difficult to catch errors.
>>
>> --Leo--
>>
>
> This answer, while technically correct, is actually misguided.
>
> 1/3 == 1/3 is guaranteed to be true. It's the same expression. (1/6 +
> 1/6 == 1/3 is a different question.)
>
> But when you're discussing SERIALIZING floats, %.14g is not
> sufficient. %.16g is sufficient to uniquely identify every possible
> double-precision IEEE floating point number with a clean round trip.

'%.16g' is not sufficient to uniquely identify every possible double
precision IEEE floating point number because base-2 floats are
not compatible with base-10 floats at this precision.
Here is a simple contra-example

In Lua-5.1 and LuaJIT (git HEAD) under Linux x86 running the following function

function eql(x)
    local fmt= '%.16g'
    local str= fmt:format(x)
    print(x == tonumber(str), str)
end

eql(1/6) ==> false   0.1666666666666667

Changing format to '%.17g' seems to work, though.

>
> So there IS a valid question to be asked here: Why SHOULDN'T
> tostring() use %.16g?
>

"%.16g" is not good enough but "%.17g" seems to pass conversion test
for few cases I ran.

--Leo--