Re: Floating point inaccuracies

[HyperHacker <hyperhacker@...>]

On Sun, Nov 28, 2010 at 15:43, Louis Mamakos <louie@transsys.com> wrote:
>
> On Nov 28, 2010, at 5:41 PM, Andrew Lentvorski wrote:
>
>> On 11/28/10 1:28 PM, Patrick Rapin wrote:
>>> This sort of problems, not directly related to Lua but to the IEEE 754
>>> floating point standard, comes up regularly to the list.
>>> Wouldn't be possible to find a more generic solution?
>>
>> 1) Switch to Decimal Floating Point: http://speleotrove.com/decimal/
>>
>> 2) Cram Goldberg down the throat of everyone who asks:
>> "What Every Computer Scientist Should Know About Floating Point Arithmetic"
>> http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.22.6768
>>
>> Is that generic enough? :-)
>>
>>
>> Oddly, Decimal Floating Point works very well on 32-bit embedded systems without floating point units (like ARM or MIPS).  A decimal floating point library is generally no slower (and often faster) than a corresponding binary floating point library and converting to/from text requires way less code, less working memory, less uncertainty and is generally faster to boot.
>>
>> -a
>>
>
> and perhaps http://luaforge.net/projects/ldecnumber/
>
>
>

I suppose you could call a function to set the number of decimal
places of accuracy to use when comparing numbers, with nil (default)
meaning a standard compare with the usual caveats.

Of course that involves modifying the language at a fairly low level,
and so you might be better off doing a string compare wherever you
need it:
> x = 1
> for i = 1,5 do x = x + 0.2 end
> =(x==2)
false
> =(("%1.4f"):format(x) == '2.0000')
true
> =("%1.20f"):format(x)
1.99999999999999977796

Of course better still is to just not use == with floating-point
numbers and learn why.

-- 
Sent from my toaster.