Re: why 0.1+0.2-0.3~=0

[Ross Bencina <rossb-lists@...>]

On 13/03/2012 7:56 PM, Dirk Laurie wrote:
> Because 0.1 is not exactly 1/10, 0.2 is not exactly 2/10,
> 0.3 is not exactly 3/10.  Those "not exactly" terms
> sometimes cancel out but usually do not.  You can
> can see it if you print out more than 16 digits.
>
> > >  print(string.format("%.17f",0.1))
> 0.10000000000000001
> > >  print(string.format("%.17f",0.2))
> 0.20000000000000001
> > >  print(string.format("%.17f",0.3))
> 0.29999999999999999


Beyond the issue of finite floating point precision, there is also the 
non-trivaial issue of converting losslessly between strings and floats.

Out of curiosity, how does Lua perform its float<->string conversions?

I would have expected a conversion from "0.1" to double and back to 
string to be lossless, even when printing extra digits. I admit a large 
degree of ignorance here, but I have read the papers cited below.

Does Lua depend on (often poor) native sprintf implementations? or does 
it use some variant of David Gay's fp routines?

http://portal.acm.org/citation.cfm?id=93557
http://portal.acm.org/citation.cfm?id=989431
http://netlib.sandia.gov/fp/

Ross.