Re: string.format("%x")

[Mike Pall <mikelu-0711@...>]

Luiz Henrique de Figueiredo wrote:
> Eric Tetz wrote:
> > string.format("%x", 4294964480)
> > string.format("%x", 0xFFFFF500)
> > 
> > Produce "80000000" rather than "FFFFF500".
> 
> This looks like a bad conversion to int. Check lua_number2int in luaconf.h.
> Forcing the cast may work.

The above is a valid unsigned int, but not a valid int. And the
number2int macro is not used for string.format (but see below):

  sprintf(buff, form, (unsigned LUA_INTFRM_T)luaL_checknumber(L, arg));

So it looks like MSVC7 has a problem casting doubles to unsigned
32 bit ints? It really has to convert to a 64 bit int and then
ignore the upper 32 bits.

Try to compile this:

  unsigned int foo(double x) { return (unsigned int)x; }

And check the assembler output. It should contain:

  fistp qword ptr [<address>]  // Intel syntax
-or-
  fistpll <address>            // GCC syntax

Note the difference between a qword (64 bit) and a dword (32 bit)
store (or fistpll vs. fistpl in the ugly GCC x86 syntax). It's a
compiler bug if a 32 bit store is used.


[
A general warning about lua_tointeger() (which uses the
number2int() macro internally):

Both the fld/fistp solution for MSVC and the int cast (slow on
x87) may return an undefined result if used on numbers outside
the signed integer range. The x87 FPU always returns 0x80000000.
Other FPUs may return any other value, even random bits.

The bias+union trick which is used with GCC on x87 gives the
desired result for the full range of signed and unsigned ints as
a side-effect.

Unfortunately any library using lua_tointeger() may produce
different results depending on the compiler, if it tries to
explicitly or implicitly treat the result as an unsigned int.
Be extra careful with bit operations and the like.

Library writers really need to use (unsigned int)lua_tonumber()
to get predictable results.
]

--Mike