Re: why tostring for number use LUA_NUMBER_FMT "%.14g", not "%.16g" ?

[Coda Highland <chighland@...>]

On Mon, Sep 9, 2013 at 7:20 PM, Leo Razoumov <slonik.az@gmail.com> wrote:
> On 9/8/13, Coda Highland <chighland@gmail.com> wrote:
>> On Sat, Sep 7, 2013 at 1:51 PM, Leo Razoumov <slonik.az@gmail.com> wrote:
>>> On 9/7/13, pulleyzzz_gmail <pulleyzzz@gmail.com> wrote:
>>>> for this code:
>>>>
>>>> a=tostring(1/3)
>>>> print(1/3==tonumber(a))  -- false
>>>>
>>>> --if use .16g
>>>>
>>>> a=string.format('%.16g',1/3)
>>>> print(1/3==tonumber(a))  -- true
>>>>
>>>
>>> Changing format to "%.16g" would not solve your problem of comparing
>>> floats with == operator.
>>>
>>> 1/3 is computed in Lua natively in binary representation.
>>>
>>> a=string.format('%.16g',1/3) is a binary representation converted to a
>>> finite length
>>> base-10 number. Very often it cannot be done without loss of accuracy.
>>> Next, you are converting it to binary again with tonumber(a) which
>>> causes another loss of accuracy.
>>>
>>> As a rule of thumb do not compare floats with == operator. It will
>>> lead to subtle and
>>> difficult to catch errors.
>>>
>>> --Leo--
>>>
>>
>> This answer, while technically correct, is actually misguided.
>>
>> 1/3 == 1/3 is guaranteed to be true. It's the same expression. (1/6 +
>> 1/6 == 1/3 is a different question.)
>>
>> But when you're discussing SERIALIZING floats, %.14g is not
>> sufficient. %.16g is sufficient to uniquely identify every possible
>> double-precision IEEE floating point number with a clean round trip.
>
> '%.16g' is not sufficient to uniquely identify every possible double
> precision IEEE floating point number because base-2 floats are
> not compatible with base-10 floats at this precision.
> Here is a simple contra-example
>
> In Lua-5.1 and LuaJIT (git HEAD) under Linux x86 running the following function
>
> function eql(x)
>     local fmt= '%.16g'
>     local str= fmt:format(x)
>     print(x == tonumber(str), str)
> end
>
> eql(1/6) ==> false   0.1666666666666667
>
> Changing format to '%.17g' seems to work, though.
>
>>
>> So there IS a valid question to be asked here: Why SHOULDN'T
>> tostring() use %.16g?
>>
>
> "%.16g" is not good enough but "%.17g" seems to pass conversion test
> for few cases I ran.
>
> --Leo--
>

Yeah, I know, I'd made that correction several posts ago. >.>

/s/ Adam