lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

On Thu, Dec 28, 2006 at 08:05:53PM -0500, Joe Smith wrote:
> No. What they are warning is that
> x=y;
> assert(x==y);
> may fail, even for !isnan(y)
> Lets say y is a 10-byte extended precision double in the FPU.
> For some reason, the compiler decides that 'x' should be storred in memory.
> So x is set to a 8-byte representation of y, created by truncation. The 
> extra 2 bytes of precision means that
> x and y do not compare equally.

We all know the issues with the x86 high-resolution FPU registers;
we don't need an introduction.  :)  That's not what it says, though:

> For example, you can find that a floating point value which is not a
> NaN is not equal to itself.

This is saying that assert(x==x) may fail for non-NaN, and seems to be
specifically saying that the "x!=x" test for NaN-ness may not always
work.  As far as I know, this is simply incorrect, and this documentation
is wrong.

Putting it another way, if assert(x==y) fails (and neither one is NaN),
then x and y are not the same value, and you're not comparing a value to
itself, so this isn't what the above is talking about.  If you do "y = x",
and y is truncated from x, then you're not assigning the same value at
all; you're assigning an approximation, a different value.

Glenn Maynard