Re: Rounding errors?

[Christian Vogler <cvogler@...>]

On Wed, Sep 18, 2002 at 10:33:54AM -0700, Ian Latham wrote:
> I don't know, I might expect some inaccuracy when I'm dealing with
> numbers that have 8 or 9 significant digits, but being unable to
> accurately represent a float to 2 decimal points seems ridiculously
> inaccurate to me.

First of all, it is meaningless to talk about accuracy to decimal
points. Floating point numbers are stored in binary, not decimal. All
and any accuracy that you talk about can only refer to the binary
digits.

All the numbers that you displayed are perfectly accurate to the first
8 or 9 binary digits. That is not what your problem is. The problem is
that these binary numbers cannot store an accurate representation of
0.01. Only sums of powers of two are accurate. In binary, 0.01 becomes

1/2^4 + 1/2^5 + 1/2^8 + 1/2^9 + 1/2^12 + 1/2^13 + ...

which is an infinite fraction with period 4. 

As a result, the least significant digits of the sums will contain
errors. When you convert them to decimal, and if the error propagates
within the number of significant digits in the decimal conversion,
they do show up in the decimal number. Worse, the error will propagate
and eventually overwhelm the computation if you just keep summing long
enough.

This is not a bug in tostring(). Its conversion is accurate to many
digits. The problem is that the underlying binary number is not. If
you want "accuracy" to 2 decimal points, you have to round to the 2nd
digit after the decimal point. 

However, note that doing so only hides the fundamental problem. The
moral of the story is, if you need accurate numbers, either work only
with sums of powers of two, or use exact arithmetic instead of
floating point arithmetic.

Regards
- Christian