lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

We are also using 24 bit precision because of DX and I converted Lua to use floats. It seems that it's actually Lua that is trashing the FPU in certain cases from the VM. Perhaps there is some casting to/from float between script tokens or something that is causing this?  I don't have the time to track it down atm, but I will try to find the exact cause, meanwhile wrapping the calls into and out of Lua fixes the problem (which is the FPU nan bit gets set when a read occurs on an uninited register).

----- Original Message ----- 
From: "John Paquin" <>
To: "Lua list" <>
Sent: Tuesday, March 09, 2004 1:10 AM
Subject: RE: control that floating point [was: Re: single precision]

> This is a little off topic, but I thought I'd add a little note on
> precision.
> If you're using Direct3d, the default behavior will *definitely* trash the
> FPU status word (specifically, it changes the precision to 24 bits for
> speed).  If you're using doubles, you *must* init your d3d device with
> I chose to change Lua to use floats rather than change the FPU back and
> forth.