[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
**Subject**: **RE: Rounding errors?**
**From**: RLake@...
**Date**: Wed, 18 Sep 2002 13:02:11 -0500

> I don't know, I might expect some inaccuracy when I'm dealing with
> numbers that have 8 or 9 significant digits, but being unable to
> accurately represent a float to 2 decimal points seems ridiculously
> inaccurate to me. That's why I wonder if it's just a bug in the tostring
> function.
You are probably working on a machine which stores floating point numbers
in binary format, not in decimal format. :) If you tried the same thing
with 1/256 instead of 1/100, it would be accurate; unfortunately, 0.01 does
not have an accurate binary representation, just like 1/3 does not have an
accurate decimal representation.
You might also want to compare your results with the following:
x=0
upper_bound = 100
while x<upper_bound do
x=x+1
print(x/100)
end
In general, Fortran-derived DO loops do suffer from floating-point drift on
many architectures (although there are known solutions to this problem, and
it will work correctly on Intel hardware if the control variable is kept in
a floating-point register, because the additional 16 bits correct for the
rounding errors.) In general, it is better to translate such loops into
something like the following:
local repetitions = floor( (upper_bound - lower_bound) / increment )
for i = 0, repetitions do
local x = lower_bound + i * increment
print(x)
end
The above code is not perfect (it needs to check for the case where
upper_bound is less than lower_bound, for example) but it is indicative.
There will be objections about the "extra work" done by the computation of
x; in most modern architectures, however, there is a "floating multiply and
add" primitive which does the work quite rapidly. (Lua does not use this
primitive because it is, sadly, not available in standard C.)