lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


On 2010-01-09, Rene Rebe <rene@exactcode.de> wrote:
>  Actually your bitop lib has very strange rounding characteristics due to the exotic number conversion:
>
>  > return  bit.bor(1000.5, 0)
>  1000
>  > return  bit.bor(1000.6, 0)
>  1001
>

Decimal number 1000.6 does not have a finite binary representation
and, thus, cannot usefully participate in bitwise operations. As a
rule of thumb do bitwise ops only on integers. Mike Pall has an
explicit paragraph on this rule in BitOP documentation.
http://bitop.luajit.org/semantics.html
"Bitwise operations cannot sensibly be applied to FP numbers (or their
underlying bit patterns). They must be converted to integers before
operating on them and then back to FP numbers."

--Leo--