lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]




On Fri, Dec 12, 2008 at 3:58 PM, Doug Currie <doug.currie@gmail.com> wrote:

related. I suspect this stems from a double to int32_t conversion. Since 0x0000ffffffff is outside the range of int32_t the conversion is returning an "integer NaN." In other words, the conversion is assumed to be signed, whereas the value you "want" is unsigned.

Searching the MS site, I found

Conversions from float, double, or long double values to unsigned long are not accurate if the value being converted is larger than the maximum positive long value. 

http://msdn.microsoft.com/en-us/library/d3d6fhea(VS.71).aspx

So, it seems that TOBIT needs a different approach.