[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: why tostring for number use LUA_NUMBER_FMT "%.14g", not "%.16g" ?
- From: Coda Highland <chighland@...>
- Date: Mon, 9 Sep 2013 21:23:26 -0700
On Mon, Sep 9, 2013 at 7:20 PM, Leo Razoumov <email@example.com> wrote:
> On 9/8/13, Coda Highland <firstname.lastname@example.org> wrote:
>> On Sat, Sep 7, 2013 at 1:51 PM, Leo Razoumov <email@example.com> wrote:
>>> On 9/7/13, pulleyzzz_gmail <firstname.lastname@example.org> wrote:
>>>> for this code:
>>>> print(1/3==tonumber(a)) -- false
>>>> --if use .16g
>>>> print(1/3==tonumber(a)) -- true
>>> Changing format to "%.16g" would not solve your problem of comparing
>>> floats with == operator.
>>> 1/3 is computed in Lua natively in binary representation.
>>> a=string.format('%.16g',1/3) is a binary representation converted to a
>>> finite length
>>> base-10 number. Very often it cannot be done without loss of accuracy.
>>> Next, you are converting it to binary again with tonumber(a) which
>>> causes another loss of accuracy.
>>> As a rule of thumb do not compare floats with == operator. It will
>>> lead to subtle and
>>> difficult to catch errors.
>> This answer, while technically correct, is actually misguided.
>> 1/3 == 1/3 is guaranteed to be true. It's the same expression. (1/6 +
>> 1/6 == 1/3 is a different question.)
>> But when you're discussing SERIALIZING floats, %.14g is not
>> sufficient. %.16g is sufficient to uniquely identify every possible
>> double-precision IEEE floating point number with a clean round trip.
> '%.16g' is not sufficient to uniquely identify every possible double
> precision IEEE floating point number because base-2 floats are
> not compatible with base-10 floats at this precision.
> Here is a simple contra-example
> In Lua-5.1 and LuaJIT (git HEAD) under Linux x86 running the following function
> function eql(x)
> local fmt= '%.16g'
> local str= fmt:format(x)
> print(x == tonumber(str), str)
> eql(1/6) ==> false 0.1666666666666667
> Changing format to '%.17g' seems to work, though.
>> So there IS a valid question to be asked here: Why SHOULDN'T
>> tostring() use %.16g?
> "%.16g" is not good enough but "%.17g" seems to pass conversion test
> for few cases I ran.
Yeah, I know, I'd made that correction several posts ago. >.>