[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
**Subject**: **Re: numerical approximations can suck!**
**From**: David Jones <drj@...>
**Date**: Fri, 21 Nov 2003 14:23:08 +0000

On Nov 20, 2003, at 20:06, Andy Jones wrote:

Hypothetical rant: we've been computing as a species for 60+ years?!?1
can't we have accurate math!?!?!

`It's called "rounding error" and I'm afraid it's due to the
``limitations of how we store decimal numbers. In some senses rounding
``error can be said to be a product of decimal arithmetic itself. So I
``think we're stuck with it.
`

`When using integers (I mean the natural whole numbers, not any
``language-specific type) there is not rounding error even if performing
``computations using floating-point arithmetic on a computer.
`

`The result of 1 * 100 + 5 * 10 + 0 is exactly 150 even when using
``floating-point arithmetic.
`
I also cannot recommend "What Every Computer Scientist Should Know About
Floating-Point Arithmetic" too highly.
http://docs.sun.com/source/806-3568/ncg_goldberg.html

`Lua can bring out problems with the implementation of strtod, for
``converting from a printed decimal form to the internal floating point
``representation, and printf, for converting from floating point to
``printed form, on some platforms. Basically some platforms get the
``wrong answers even for apparently simple cases like strtod("113") or
``printf("%17d", 14). There is no excuse for this (it's the fault of the
``provider of the system C libaries usually) but it can be hard for users
``to do anything about it.
`
drj