RE: numerical approximations can suck!

[jmckenna@...]

I can't recommend "What Every Computer Scientist Should Know About
Floating-Point Arithmetic" too highly.

http://docs.sun.com/source/806-3568/ncg_goldberg.html

Since Lua floating point as its only numeric type (by default), it's more
important for Lua programmers to be familiar with the issues.

> -----Original Message-----
> From: Brett Bibby [mailto:research@gamebrains.com] 
> Sent: 20 November 2003 10:37
> To: Lua list
> Subject: Re: numerical approximations can suck!
> 
> 
> What he said, but do realize that once you fix that problem 
> 0.0 can be signed, nan, or a number of other things that may 
> make you more unhappy than you are now :-)  I highly 
> recommend you study the IEEE floating point specification as 
> they were well aware of the issues involved and had to make a 
> number of tradeoffs in implementation that could frustrate 
> you.  Also, you can redefine what a lua number is if you want to....
> 
> ----- Original Message ----- 
> From: <jmckenna@reflectionsinteractive.com>
> To: <lua@bazar2.conectiva.com.br>
> Sent: Thursday, November 20, 2003 5:11 PM
> Subject: RE: numerical approximations can suck!
> 
> 
> > Either your computer's floating point is broken, or x and y are
> genuinely
> > slightly different.  Floating point calculations are exact for
> integers (as
> > long as you're in range.  The range of double precision 
> floating point
> is
> > larger than that of 32 bit integers, so that's not usually 
> a problem).
> >
> > x = 150
> > y = 150
> > print( x, y, x-y )
> > -> 150  150  0
> > x = 150 + 1.9895196601283e-13
> > y = 150
> > print( x, y, x-y )
> > -> 150  150  1.9895196601283e-13
> >
> > It could be print's helpful truncating of numbers that's confusing
> you.
> >
> > math.floor and math.ceil might be the functions you're looking for,
> but I
> > suspect you're blaming the wrong thing.
> >
> > > -----Original Message-----
> > > From: Ando Sonenblick [mailto:ando@spritec.com]
> > > Sent: 20 November 2003 07:19
> > > To: Lua list
> > > Subject: numerical approximations can suck!
> > >
> > >
> > > Gang,
> > >
> > > I've recently begun to lament all floating point (be it floats or 
> > > doubles).
> > >
> > > Since when does 150 - 150 equal 1.9895196601283e-13?
> > >
> > > My lua code has dx = x - y and print(x); print(y); 
> print(dx) results 
> > > in:
> > >
> > > 150
> > > 150
> > > 1.9895196601283e-13
> > >
> > > Now I know that to you and mean (and to integer arithmetic) 
> > > 1.9895196601283e-13 is effectively zero, but it's somehow screwing
> up
> > > my math and my programs functionality!
> > >
> > > Hypothetical rant: we've been computing as a species for 60+ 
> > > years?!?1 can't we have accurate math!?!?!
> > >
> > > real world question: is there a way to convert a lua number (ie 
> > > double) into its integer counterpart?  I'm simply (currently) 
> > > working with integer precision (or WANT to be doing so)...
> > >
> > > thx,
> > > ando "don't even MENTION trying to work with numtostring 
> and floats 
> > > with values of 1.9895196601283e-13" moon :)
> > >
>