[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: numerical approximations can suck!
- From: Ando Sonenblick <ando@...>
- Date: Wed, 19 Nov 2003 23:18:58 -0800
Gang,
I've recently begun to lament all floating point (be it floats or
doubles).
Since when does 150 - 150 equal 1.9895196601283e-13?
My lua code has dx = x - y and print(x); print(y); print(dx) results in:
150
150
1.9895196601283e-13
Now I know that to you and mean (and to integer arithmetic)
1.9895196601283e-13 is effectively zero, but it's somehow screwing up
my math and my programs functionality!
Hypothetical rant: we've been computing as a species for 60+ years?!?1
can't we have accurate math!?!?!
real world question: is there a way to convert a lua number (ie double)
into its integer counterpart? I'm simply (currently) working with
integer precision (or WANT to be doing so)...
thx,
ando "don't even MENTION trying to work with numtostring and floats
with values of 1.9895196601283e-13" moon :)