[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
**Subject**: **Re: math.floor and rounding**
**From**: James Graves <ansible@...>
**Date**: Mon, 20 Sep 2010 19:06:37 -0500

On Mon, Sep 20, 2010 at 11:53:47PM +0800, KHMan wrote:
> On 9/20/2010 11:32 PM, Jose wrote:
> > I want to round numbers with two decimal digits.
> >
> > I use math.floor( n * 100 ) but sometimes I have situations like:
> >
> > n = 10.20
> > math.floor( n * 100 ) --> 1019
> >
> > My current solution is to always add 0.001 to n, but is this the right
> > approach ?
>
> If it's *precisely* 2 decimal places, consider instead to do your
> calculations without any fractions, i.e. use 1020 etc.
If you want to _round_ off a floating point number, instead of
flooring it, here is the rounding function I have in my standard
library:
function round(num) return math.floor(num+.5) end
It works better when you've got more digits of precision than you
wanted:
n = 10.2049
round(n * 100) --> 1020
n = 10.2051
round(n * 100) --> 1021
Are you handling negative numbers? math.floor may not be what you want
in that case:
n = -30.000001
math.floor(n * 100) --> -3001
round(n * 100) --> -3000
I encourage you to read up on round, floor and ceiling:
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
It is my believe that what people usually want when converting a
floating point number to an integer is the nearest integer. That
depends on the domain, obviously. So if your doing accounting, the
rules a different than if you're just drawing some graphics on the
screen.
James Graves