[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: Floating point inaccuracies
- From: Patrick Rapin <toupie300@...>
- Date: Sun, 28 Nov 2010 22:28:07 +0100
This sort of problems, not directly related to Lua but to the IEEE 754
floating point standard, comes up regularly to the list.
Wouldn't be possible to find a more generic solution?
The default conversion from a number to a string, used notably in
print function, uses 16 digits of precision (based on sprintf("%.16g")
).
So my idea is the following: if two very close numbers only differ
from their 17th digit or after, the == operator between them should
return true.
I know that this is much more difficult to implement and slower than a
simple binary comparison; maybe it would be too complicated to a fast
embedded language like Lua.
But wouldn't be better than the current situation?