Re: passing lua numbers to a C function that takes float values

[KHMan <keinhong@...>]

Aaron Saarela wrote:
> Thanks to everyone that helped out with my yield problem earlier. I’ve
> reworked my app and solved that problem.
> 
> My next problem is related to passing float values between C++ and Lua.
>  I have a number in Lua (“8.2008” in particular).  If I call another Lua
> function with this value, it remains consistent. When I call a C
> function that takes a float with this value, however, it’s received as
> ‘8.2007999999999992”.  If I pass this value back to Lua (using
> lua_pushnumber), it becomes ‘8.2007999420166’.  And so on. I’m loosing
> precision on each Lua / C transition.
> [snip]
> Is this just a side-effect of the conversion between double and float or
> is there a way around this?

floats have a 24 bit mantissa.
doubles have a 53 bit mantissa.

The number of significant digits is approximately:
  log10(2) * (size of mantissa)

This gives (around) 7 for float and 16 for double. For C on i386,
unless you have specific needs and you know exactly what you are
doing (e.g. for extreme size/performance needs), do keep to
doubles and switch to float only at the edges or for end results.

If you switch back-and-forth, precision will be lost. The above is
still 8.200800 -- it is still 'correct' based on what you have
done. Depending on the application, this behaviour may or may not
be acceptable.

-- 
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia