Re: Floating point inaccuracies

[Axel Kittenberger <axkibe@...>]

been awhile dunno if I recall it correctly, but what you learn in
coding 101 is something like this. Correct me if the comparisons are
wrong. MYDELTA is an arbitrary tolerance value, I gave a random
number, depending on how much crazy calculation you did before, until
you still consider two values equal. Maybe better code adapts delta on
the size of a/b.

MYDELTA = 0.0000000001

function equal(a, b)
    return a+MYDELTA >= b and a-MYDELTA <= b
end

On Mon, Nov 29, 2010 at 12:39 AM, Tony Finch <dot@dotat.at> wrote:
> On 28 Nov 2010, at 22:41, Andrew Lentvorski <bsder@allcaps.org> wrote:
>>
>> Switch to Decimal Floating Point: http://speleotrove.com/decimal/
>
> That just moves the problem from 1/10 to 1/3 etc.
>
> Tony.
> --
> f.anthony.n.finch  <dot@dotat.at>  http://dotat.at/
>>
>
>