Re: numerical approximations can suck!

[David Jones <drj@...>]

On Nov 20, 2003, at 20:06, Andy Jones wrote:

> > Hypothetical rant: we've been computing as a species for 60+ years?!?1
> > can't we have accurate math!?!?!
>
> It's called "rounding error" and I'm afraid it's due to the 
> limitations of how we store decimal numbers.  In some senses rounding 
> error can be said to be a product of decimal arithmetic itself.  So I 
> think we're stuck with it.

When using integers (I mean the natural whole numbers, not any 
language-specific type) there is not rounding error even if performing 
computations using floating-point arithmetic on a computer.

The result of 1 * 100 + 5 * 10 + 0 is exactly 150 even when using 
floating-point arithmetic.

I also cannot recommend "What Every Computer Scientist Should Know About
Floating-Point Arithmetic" too highly.

http://docs.sun.com/source/806-3568/ncg_goldberg.html


Lua can bring out problems with the implementation of strtod, for 
converting from a printed decimal form to the internal floating point 
representation, and printf, for converting from floating point to 
printed form, on some platforms.  Basically some platforms get the 
wrong answers even for apparently simple cases like strtod("113") or 
printf("%17d", 14).  There is no excuse for this (it's the fault of the 
provider of the system C libaries usually) but it can be hard for users 
to do anything about it.

drj