[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: [ANN] Lua 5.2.0 (work1) now available
- From: David Kastrup <dak@...>
- Date: Sat, 09 Jan 2010 13:54:17 +0100
Rene Rebe <rene@exactcode.de> writes:
> Actually your bitop lib has very strange rounding characteristics due to the exotic number conversion:
>
>> return bit.bor(1.4, 0)
> 1
>> return bit.bor(1.5, 0)
> 2
>> return bit.bor(1000.5, 0)
> 1000
>> return bit.bor(1000.6, 0)
> 1001
Round to even is standard IEEE rounding behavior. Causes the smallest
mean error for cumulative rounding operations and bias of 0.
--
David Kastrup
