[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

• Subject: Re: Floating point inaccuracies
• From: Patrick Rapin <toupie300@...>
• Date: Tue, 30 Nov 2010 15:29:58 +0100

```> One problem I thought of immediately was comparison to zero, which is
> usually intended to be a check for exactly zero.

This is true, but it is not in conflict with my proposition.
A number close to 0 but different from 0 has a very different
representation, with plenty of significant digits before the exponent
part.
Example:
print(math.exp(-740)) --> 4.1995579896506e-322

> But, there is an even bigger problem with approximate comparisons:  Sort
> order would break [assuming approximation applies to less/greater in
> addition to equals].

It is effectively logical to make the approximation also on
comparison, I also had that idea in mind.
This could break the sort order in a table, that's right. I am not
sure that this is such an issue (sorting extremely close numbers is
not so common, and probably not very useful), but anyway, I may
suggest a solution.
Lua has distinct metatable entries and opcodes for operator less-than
( < ) and less-than-or-equal ( <= ), although that the second could be
expressed in terms of the first. From Lua documentation:
"Note that, in the absence of a "le" metamethod, Lua tries the "lt",
assuming that a <= b is equivalent to not (b < a). "

What if only <= operator performs the approximation, and not <
operator? The sort function will use <, and applications accepting the
approximation may use <= or its symmetrical form >= .

```