Re: control that floating point [was: Re: single precision]

["Brett Bibby" <research@...>]

We are also using 24 bit precision because of DX and I converted Lua to use floats. It seems that it's actually Lua that is trashing the FPU in certain cases from the VM. Perhaps there is some casting to/from float between script tokens or something that is causing this?  I don't have the time to track it down atm, but I will try to find the exact cause, meanwhile wrapping the calls into and out of Lua fixes the problem (which is the FPU nan bit gets set when a read occurs on an uninited register).
Brett

----- Original Message ----- 
From: "John Paquin" <jpaquin@breakawaygames.com>
To: "Lua list" <lua@bazar2.conectiva.com.br>
Sent: Tuesday, March 09, 2004 1:10 AM
Subject: RE: control that floating point [was: Re: single precision]


> This is a little off topic, but I thought I'd add a little note on
> precision.
> 
> If you're using Direct3d, the default behavior will *definitely* trash the
> FPU status word (specifically, it changes the precision to 24 bits for
> speed).  If you're using doubles, you *must* init your d3d device with
> FPU_PRESERVE.
> 
> I chose to change Lua to use floats rather than change the FPU back and
> forth.
> 
>