Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

[sur-behoffski <sur_behoffski@...>]

G'Day,

[Sorry for not replying within the relevant thread, but I read the list
use digest mode.  Apologies if this offends anyone.]

I managed publish an article in the November 1997 edition of Dr. Dobb's
Journal.  In that same edition, was an extended interview with Prof.
William Kahan, entitled "A Conversation with William Kahan":

        http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/1997/9711/9711a/9711a.htm

IMHO, the entire Kahan article is well worth reading.

I'll pick out a couple comments about the difference between abstract/simple
formulas, versus complicated, but numerically stable algorithms, and end
with a longish extract about different rounding modes that the standard
lets the program(mer) select, and why being able to select the rounding
mode at runtime may be valuable:

--

        [...]
        DDJ: What should we programmers be learning?

        WK: What you should be learning are things from numerical-analysis
        classes, such as why the less accurate of two ways of calculating a
        function may be perfectly satisfactory for engineering work.  But I
        also tell programmers that argument [such as on the phone with
        customers] takes time and money, and since we know a way to do it
        which will circumvent the necessity for argument, by all means do
        it that way, if you can. You can dispense with the necessity of
        trying to persuade somebody of something which is, in fact, true --
        that two methods are just as good as one another, but which they
        have every right to disbelieve. There are all sorts of folks who
        tell you, "It's okay, this is just as good," and are wrong.

        Let's calculate accuracy well out of proportion with what anybody
        actually needs. They don't need it, but it doesn't matter. We'll do
        it that way and then there won't be any argument.

        [...]

--

About the difference between "intellectually economical" and simple:

--
        [...]

        DDJ: So what should our industry be doing?

        WK: I think it was Churchill who once said something about the
        essence of leadership being figuring out where people are going to
        go anyway and getting there ahead of them. The industry, in its
        own best interests, has to take a role that leads the customers,
        but not by too much. When it comes to numerical things, part of
        leadership is to provide an intellectually economical numerical
        ambiance.

        "Intellectually economical" doesn't necessarily mean simple. We
        have to understand the various arenas in which people want to
        depend upon computers. We have to understand them in some ways
        better than the customers. The customers don't always understand
        the issues because they are preoccupied with what matters to
        them, not with the design and construction of the systems upon
        which they depend.

        It's like being engaged in sewer repair. The art of sewer repair
        is to do it in such a fashion so as not to oblige others to think
        about it at all. If you have to think about it, the sewer
        repairmen haven't been doing their job.

        DDJ: So we programmers are not doing our job?

        WK: That's correct. It is straightforward to implement numerics
        better. I've written about it in a number of places. Look at my
        article about miscalculating the area and angles of a triangle at:

                http://http.cs.berkeley.edu/~wkahan/Triangle.ps

        There's an abstract there that points to a number of
        misconceptions or superstitions. It doesn't seem to me to be all
        that difficult for the compiler community to serve these needs.

        If the industry gets far enough down a certain path, it can't
        turn, never mind go back! [...]

--

And, finally, some specific comments on why multiple different rounding
modes were mandated in the IEEE standard:

--

        WK: [...] But if you get rid of a [preprocessor] expression when
        someone has written it in cold blood, you're changing the
        semantics of his program.

        Similar are things like constant folding. You (the optimizer)
        look at an expression and decide, "Oh, this is a constant, so I
        can compute it at compile time." But maybe that expression was
        designed for a different rounding mode at run time than the one
        which prevails at compile time. There are reasons to change the
        rounding mode without mentioning it.

        There's all sorts of stuff in the IEEE Standard designed to help
        ordinary people do things like diagnose what may be screwing
        them in a module they got from someone else. Perhaps there's an
        algorithm that is pretty good for almost all data except yours.
        It doesn't know you personally, you understand! Your data just
        happens to be the kind that that particular algorithm doesn't
        like. How do you find out if this is the case?

        One way is to change the rounding mode, which will create
        different rounding errors. This identifies modules where it
        might be worth your time to investigate. If you didn't have this
        ability to change the rounding mode, you might not have any other
        way to identify which module among many supplied by
        multiple vendors was the likely candidate for further
        investigation.

        In any army, there's a soldier who doesn't get the message.  DEC
        Alpha decided to put the rounding mode not in a control word
        alone, but also in the opcode. That means that when you compile,
        you can compile the rounding modes in such a way that they don't
        respond to a change of control word setting, and therefore, when
        you rerun the code with the changed control word, you're going
        to get the same results as before.

        It isn't that this was a bad idea, to put the rounding mode in
        the opcode as well as the control word. The mistake was that they
        put the default rounding mode into the opcode. The default should
        have been left in the control word.

        [...]

--

I have to sheepishly admit that 99.9% of Numerical Methods are above my
head, but I'm glad that I've come to know about Prof. Kahan's work and
his very careful, but also accessible, writings about computerised
numerical computation.

I hope that this post is of value in the context of the recent/ongoing
discussion(s) about rounding modes, and their interaction with proposals
to set rounding modes for single/double/extended precision in Lua.

cheers,

sur-behoffski (Brenton Hoff)
programmer, Grouse Software