[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: weird bug?
- From: Alan Watson <a.watson@...>
- Date: Wed, 28 Feb 2001 17:14:50 -0600
> In Lua, "print" always outputs enough digits so that the
> number may be reproduced from the textual form (16 for
I'm not sure that 16 digits is enough.
First, doubles might be bigger than IEEE doubles. This is
not a problem, because ANSI C provides a constant DBL_DIG in
<float.h> that is defined as the "number of decimal digits,
q, such that any floating point number with q decimal digits
can be rounded into a floating point number ... and back
again without change to the q decimal digits". For an IEEE
double it is actually 15.
However, even if we make the assumption that doubles are
IEEE doubles, 15 (or 16) digits is not when %g is used. If
you use %g, you need a precision of at least DBL_DIG + 2.
Why? When we print with %e, the mantissa consists of an
integer part with one digit plus a fraction part whose
number of digits is equal to the precision. Thus:
printf("%.*e", DBL_DIG - 1, x);
should print x unambiguously.
However, %g uses %f if the exponent of the argument is >= 4
or < the precision. Thus, 1.0000...e-4 is printed as
0.00010000..., and so we effectively lose 4 digits of
precision. Thus, we need to use something like:
printf(f, "%.*g", DBL_DIG + 3, x);
I think this is right, but I've just had a large lunch, so
someone should check it; it's quite possible that I'm off by
a digit or two.
Dr Alan Watson
Instituto de Astronomía UNAM