lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

David Kastrup wrote:
Roberto Ierusalimschy <> writes:

on my WinXP 32 bit box as well as an Ubuntu Karmic 32-bit box.
This makes it pretty clear that the conversion from double to int
is handling the overflow in an unexpected way. Knowing it will
saturate, having -2147483649 saturate to -2147483648 is
acceptable. However having 2147483649 saturate to the same value
is *surprising*.
All values "saturate" to 0x80000000.

It seems that the positive ones should rather go towards 0x7fffffff.

No, this is not saturation arithmetic, but an x86/x86-64 architecture detail. 0x80000000 is the "indefinite integer value" for 32-bit results when converting, for example, when using CVTTSD2SI or FISTP. The indefinite integer value is a special value when the number is out of range or cannot be converted due to some reason. See the Intel/AMD docs for the details...

Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia