[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
**Subject**: **Re: Floating point inaccuracies**
**From**: HyperHacker <hyperhacker@...>
**Date**: Sun, 28 Nov 2010 17:37:57 -0700

On Sun, Nov 28, 2010 at 17:32, Axel Kittenberger <axkibe@gmail.com> wrote:
> been awhile dunno if I recall it correctly, but what you learn in
> coding 101 is something like this. Correct me if the comparisons are
> wrong. MYDELTA is an arbitrary tolerance value, I gave a random
> number, depending on how much crazy calculation you did before, until
> you still consider two values equal. Maybe better code adapts delta on
> the size of a/b.
>
> MYDELTA = 0.0000000001
>
> function equal(a, b)
> return a+MYDELTA >= b and a-MYDELTA <= b
> end
>
> On Mon, Nov 29, 2010 at 12:39 AM, Tony Finch <dot@dotat.at> wrote:
>> On 28 Nov 2010, at 22:41, Andrew Lentvorski <bsder@allcaps.org> wrote:
>>>
>>> Switch to Decimal Floating Point: http://speleotrove.com/decimal/
>>
>> That just moves the problem from 1/10 to 1/3 etc.
>>
>> Tony.
>> --
>> f.anthony.n.finch <dot@dotat.at> http://dotat.at/
>>>
>>
>>
>
>
Yep, that's a common method. Or abs(a-b) < 0.0001, but in this case
math.abs() is a function call, so maybe not as efficient?
--
Sent from my toaster.