[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)
- From: sur-behoffski <sur_behoffski@...>
- Date: Sun, 10 Jun 2018 22:24:48 +0930
[Sorry for not replying within the relevant thread, but I read the list
use digest mode. Apologies if this offends anyone.]
I managed publish an article in the November 1997 edition of Dr. Dobb's
Journal. In that same edition, was an extended interview with Prof.
William Kahan, entitled "A Conversation with William Kahan":
IMHO, the entire Kahan article is well worth reading.
I'll pick out a couple comments about the difference between abstract/simple
formulas, versus complicated, but numerically stable algorithms, and end
with a longish extract about different rounding modes that the standard
lets the program(mer) select, and why being able to select the rounding
mode at runtime may be valuable:
DDJ: What should we programmers be learning?
WK: What you should be learning are things from numerical-analysis
classes, such as why the less accurate of two ways of calculating a
function may be perfectly satisfactory for engineering work. But I
also tell programmers that argument [such as on the phone with
customers] takes time and money, and since we know a way to do it
which will circumvent the necessity for argument, by all means do
it that way, if you can. You can dispense with the necessity of
trying to persuade somebody of something which is, in fact, true --
that two methods are just as good as one another, but which they
have every right to disbelieve. There are all sorts of folks who
tell you, "It's okay, this is just as good," and are wrong.
Let's calculate accuracy well out of proportion with what anybody
actually needs. They don't need it, but it doesn't matter. We'll do
it that way and then there won't be any argument.
About the difference between "intellectually economical" and simple:
DDJ: So what should our industry be doing?
WK: I think it was Churchill who once said something about the
essence of leadership being figuring out where people are going to
go anyway and getting there ahead of them. The industry, in its
own best interests, has to take a role that leads the customers,
but not by too much. When it comes to numerical things, part of
leadership is to provide an intellectually economical numerical
"Intellectually economical" doesn't necessarily mean simple. We
have to understand the various arenas in which people want to
depend upon computers. We have to understand them in some ways
better than the customers. The customers don't always understand
the issues because they are preoccupied with what matters to
them, not with the design and construction of the systems upon
which they depend.
It's like being engaged in sewer repair. The art of sewer repair
is to do it in such a fashion so as not to oblige others to think
about it at all. If you have to think about it, the sewer
repairmen haven't been doing their job.
DDJ: So we programmers are not doing our job?
WK: That's correct. It is straightforward to implement numerics
better. I've written about it in a number of places. Look at my
article about miscalculating the area and angles of a triangle at:
There's an abstract there that points to a number of
misconceptions or superstitions. It doesn't seem to me to be all
that difficult for the compiler community to serve these needs.
If the industry gets far enough down a certain path, it can't
turn, never mind go back! [...]
And, finally, some specific comments on why multiple different rounding
modes were mandated in the IEEE standard:
WK: [...] But if you get rid of a [preprocessor] expression when
someone has written it in cold blood, you're changing the
semantics of his program.
Similar are things like constant folding. You (the optimizer)
look at an expression and decide, "Oh, this is a constant, so I
can compute it at compile time." But maybe that expression was
designed for a different rounding mode at run time than the one
which prevails at compile time. There are reasons to change the
rounding mode without mentioning it.
There's all sorts of stuff in the IEEE Standard designed to help
ordinary people do things like diagnose what may be screwing
them in a module they got from someone else. Perhaps there's an
algorithm that is pretty good for almost all data except yours.
It doesn't know you personally, you understand! Your data just
happens to be the kind that that particular algorithm doesn't
like. How do you find out if this is the case?
One way is to change the rounding mode, which will create
different rounding errors. This identifies modules where it
might be worth your time to investigate. If you didn't have this
ability to change the rounding mode, you might not have any other
way to identify which module among many supplied by
multiple vendors was the likely candidate for further
In any army, there's a soldier who doesn't get the message. DEC
Alpha decided to put the rounding mode not in a control word
alone, but also in the opcode. That means that when you compile,
you can compile the rounding modes in such a way that they don't
respond to a change of control word setting, and therefore, when
you rerun the code with the changed control word, you're going
to get the same results as before.
It isn't that this was a bad idea, to put the rounding mode in
the opcode as well as the control word. The mistake was that they
put the default rounding mode into the opcode. The default should
have been left in the control word.
I have to sheepishly admit that 99.9% of Numerical Methods are above my
head, but I'm glad that I've come to know about Prof. Kahan's work and
his very careful, but also accessible, writings about computerised
I hope that this post is of value in the context of the recent/ongoing
discussion(s) about rounding modes, and their interaction with proposals
to set rounding modes for single/double/extended precision in Lua.
sur-behoffski (Brenton Hoff)
programmer, Grouse Software