Problem when using Lua under VC.NET.

["Bilyk, Alex" <ABilyk@...>]

This post is primarily for those who use Lua with Direct X on Win32 and use VC.NET as the compiler of [under the gun at ones head] choice.

I use Lua in parallel with DirectX, which has a nasty habit for changing the FPU precision from double to single under the hood. This means that even though we use doubles, all arithmetic operations are performed with single precision. Not too bad so far. Among other things I read/write 32 bit integers with Lua. This involves no math, just storing and reading values. So, what I have is a function 

function hex2dec(hex_number_string)
	return tonumber(hex_number_string, 16)
end

And than in various places I would say

	ID1 = hex2dec('deadbeef') 
	ID2 = hex2dec('ca09f543') 

Needless to say, when I call lua_number() in C I am expecting to get exactly the number I have defined in Lua script. This worked just fine under VC6. Under VC.NET, however, ID2 would come up in C as 0xca09f500 instead of 0xca09f543. It happens that VC7 when attempting conversion from *unsigned* long to double may use floating point operations (FADD is the only one I have seen so far). Under DirectX with its switching FPU to a single precision that FADD causes the loss of the last two digits. The precise location in code that manifests the problem is in luaB_tonumber where it has 

	unsigned long n;

in the branch 

    const char *s1 = luaL_checkstring(L, 1);
    char *s2;
-->unsigned long n; /* (*) */
    luaL_argcheck(L, 2 <= base && base <= 36, 2, "base out of range");
    n = strtoul(s1, &s2, base);
    if (s1 != s2) {  /* at least one valid digit? */
      while (isspace((unsigned char)(*s2))) s2++;  /* skip trailing spaces */
      if (*s2 == '\0') {  /* no invalid trailing characters? */
-->   lua_pushnumber(L, n); /* (**) */
        return 1;
      }
    }

The fact that 'n', see (*), is *unsigned* is what seems to be causing the use of FADD and the loss of precision when calling lua_pushnumber down the road, see (**). The fix that has been working so far is to make 'n' signed, which causes the compiler to use straight FPU instructions for conversions between ints and floats without any math involved. If FPU is in double precision mode there is no problem as well. By the way, from what I can tell, this is the only place where lua_pushnumber is called with unsigned long/int value.

I don't think this is a bug in Lua per se. However, if you use Lua in parallel with Direct X in the same thread, I thought you might want to know that storing integers in Lua double precision floats is not as safe as it seemed to be... if you are compiling this with VC.NET. 

AB