[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Rounding errors?
- From: "Thomas Wrensch" <twrensch@...>
- Date: Wed, 18 Sep 2002 16:09:09 -0700
You're probably used to working in environments where this particular problem is masked by whatever the equivalent to "tostring" is in that system. Many systems automatically truncate a digit or so and round the result so your table would look correct.
You should be able to acheive exactly the same effect by using format to print fewer digits.
>>> email@example.com 09/18/02 10:15 AM >>>
Hi, I was wondering if someone could enlighten me as to why the
following script does not produce the output I would expect:
upper_bound = 1
while x<upper_bound do
I would expect to see all the values in the interval (0,.5] listed out
at .01 increments to a precision of .01, but instead I get this: