lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

On Mon, Mar 13, 2006 at 11:32:59AM -0800, Mark Manyen wrote:

> Does anyone have some ideas on how to make the floating point
> problems between DX and Lua go away that is more constructive than
> use OpenGL?

Is it perhaps the known problem where Direct3D quietly messes with the
floating point unit in ways that affect other code:

    Initializing Direct3D with default settings causes the precision
    bits in the floating-point control word to be reset such that FPU
    calculations always occur with 24-bit precision (single precision
    / float). This is much more serious as it causes roundoff errors
    to become much larger, and it means that double-precision math can
    no longer represent all valid values of a 32-bit int. For this
    reason, if you invoke Direct3D within an application that may not
    be expecting it, such as within a video filter, you should set the
    D3DCREATE_FPU_PRESERVE flag when creating the device.

Microsoft apparently has their reasons for doing this in Direct3D:

    The D3DCREATE_FPU_PRESERVE flag exists for those applications who
    want to set their own FP control word and/or stay with
    double-precision. You still must be very careful when setting it
    to non-standard rounding or enabling exceptions. For games, you
    typically don't want the performance hit of having the FP unit
    working in double-precision.


    Using this flag with the FPU in double-precision mode will reduce
    Direct3D performance.

                                                  -Dave Dodge