[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: numerical approximations can suck!
- From: Ando Sonenblick <ando@...>
- Date: Wed, 19 Nov 2003 23:18:58 -0800
I've recently begun to lament all floating point (be it floats or
Since when does 150 - 150 equal 1.9895196601283e-13?
My lua code has dx = x - y and print(x); print(y); print(dx) results in:
Now I know that to you and mean (and to integer arithmetic)
1.9895196601283e-13 is effectively zero, but it's somehow screwing up
my math and my programs functionality!
Hypothetical rant: we've been computing as a species for 60+ years?!?1
can't we have accurate math!?!?!
real world question: is there a way to convert a lua number (ie double)
into its integer counterpart? I'm simply (currently) working with
integer precision (or WANT to be doing so)...
ando "don't even MENTION trying to work with numtostring and floats
with values of 1.9895196601283e-13" moon :)