[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: numerical approximations can suck!
- From: David Jones <drj@...>
- Date: Fri, 21 Nov 2003 14:23:08 +0000
On Nov 20, 2003, at 20:06, Andy Jones wrote:
Hypothetical rant: we've been computing as a species for 60+ years?!?1
can't we have accurate math!?!?!
It's called "rounding error" and I'm afraid it's due to the
limitations of how we store decimal numbers. In some senses rounding
error can be said to be a product of decimal arithmetic itself. So I
think we're stuck with it.
When using integers (I mean the natural whole numbers, not any
language-specific type) there is not rounding error even if performing
computations using floating-point arithmetic on a computer.
The result of 1 * 100 + 5 * 10 + 0 is exactly 150 even when using
I also cannot recommend "What Every Computer Scientist Should Know About
Floating-Point Arithmetic" too highly.
Lua can bring out problems with the implementation of strtod, for
converting from a printed decimal form to the internal floating point
representation, and printf, for converting from floating point to
printed form, on some platforms. Basically some platforms get the
wrong answers even for apparently simple cases like strtod("113") or
printf("%17d", 14). There is no excuse for this (it's the fault of the
provider of the system C libaries usually) but it can be hard for users
to do anything about it.