Re: Floating point inaccuracies

[<jgiors@...>]

> -------- ORIGINAL MESSAGE ----------------------
> Date: Tue, 30 Nov 2010 15:29:58 +0100
> From: Patrick Rapin <toupie300@gmail.com>
> Subject: Re: Floating point inaccuracies
> To: Lua mailing list <lua-l@lists.lua.org>
> Message-ID:
> 	<AANLkTikoABtuK8ccQmm8hWsSDRasMoeqJf0qaw6-OBV6@mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> > One problem I thought of immediately was comparison to zero, which is
> > usually intended to be a check for exactly zero.
> 
> This is true, but it is not in conflict with my proposition.
> A number close to 0 but different from 0 has a very different
> representation, with plenty of significant digits before the exponent
> part.
> Example:
>   print(math.exp(-740)) --> 4.1995579896506e-322
> 

I don't understand your point here. My point was that when checking for
exactly zero, I would not expect my code to detect anything except zero
as zero, i.e. I expect math.exp(-740) ~= 0, but you're proposal would
make math.exp(-740) == 0. What does that have to do with the
representation? Why is it not in conflict with your proposition?

> > But, there is an even bigger problem with approximate comparisons:  Sort
> > order would break [assuming approximation applies to less/greater in
> > addition to equals].
> 
> It is effectively logical to make the approximation also on
> comparison, I also had that idea in mind.

Actually, if equality is changed, the only logical option is to change
less and less-equal, too (more below).

> This could break the sort order in a table, that's right. I am not
> sure that this is such an issue (sorting extremely close numbers is
> not so common, and probably not very useful), but anyway, I may
> suggest a solution.

Sorry to nitpick, but I think you mean "requiring that extremely close
numbers sort in the correct order is probably not very useful".
Extremely close numbers are fairly common (due to roundoff error). It
would be confusing to see them sorted out of order.

> Lua has distinct metatable entries and opcodes for operator less-than
> ( < ) and less-than-or-equal ( <= ), although that the second could be
> expressed in terms of the first. From Lua documentation:
>   "Note that, in the absence of a "le" metamethod, Lua tries the "lt",
> assuming that a <= b is equivalent to not (b < a). "
> 
> What if only <= operator performs the approximation, and not <
> operator? The sort function will use <, and applications accepting the
> approximation may use <= or its symmetrical form >= .
> 

Generally, it's a poor idea to make the operators inconsistent
(especially as a language default). Imagine how confusing it would be to
hit the assert in the following code (when x and y are numbers):

if x > y then assert( not x <= y ) end

Worse, making the operators inconsistent can make some sort algorithms
fail, possibly in an infinite loop. There are certainly countless
implementations of sort algorithms besides table.sort() out in the wild.
A change like the above would break at least some existing code.

I don't think changes to the number comparison operators can be
justified as a language default. However, since the __eq, __lt, and __le
metamethods can be overridden for numbers[*], it might be nice to create
a simple Lua library which implements epsilon comparison capability. I
don't know if there is anything like that currently available -- some
quick searches didn't turn up anything.

John Giors
Independent Programmer
Three Eyes Software
jgiors@ThreeEyesSoftware.com
http://ThreeEyesSoftware.com

[*] This is another strike against the proposal -- users can already
implement the proposed feature in Lua itself.