Re: Poor man's benchmarking issue - help needed

[Lorenzo Donati <lorenzodonatibz@...>]

On 31/08/2011 11.32, TNHarris wrote:
> Lorenzo Donati<lorenzodonatibz@interfree.it>  wrote:
> > According to the following benchmarks, built-in math.floor and
> > math.ceil
> > are slower than their counterpart rewritten using the core % operator!
> Partly because you're measuring table lookups. Alias math.floor and math.ceil to locals and the times I get go from 40% slower to just 25% slower.

Ouch! Silly me! I knew that!

My code is "littered" by local foo = lib.foo statements! :-)

My original benchmark did have locals. I did that mistake when pulling 
it out of context in order to post it :-(


> Also, that ce2 function is wrong when x%1 equals 0.

Uh! Right. I was too hasty.

Let's try again:

----------------------------------------------
local StatTime = require 'ld.debug'.StatTime

local floor = math.floor
local ceil = math.ceil

local function flo( x ) return floor( x ) end
local function flo2( x ) return x - x%1 end

local function ce( x ) return ceil( x ) end
local function ce2( x )
   local fp = x%1
   if fp == 0 then return x end
   return x - fp + 1
end


print( StatTime( 1e8, { flo, flo2 }, 5.5 ) )
  --> 2.275e-007	1	0.675
print( StatTime( 1e8, { ce, ce2 }, 5.5 ) )
  --> 2.3765e-007	1	0.745
----------------------------------------------


Still quite slower.


BTW I was contacted by PM and I got this objection:

[...]
> Your function "flo2" calculates the
> floor of a number. But the function "flo" calls another function which
> then calculates the floor. That is adding an extra function call to
> "flo", which distorts the benchmark. You should compare the two
> functions which calculate floor directly, i.e, math.floor on the one
> hand and your function flo2 on the other.
>
> Instead of this
>     print(StatTime(1e8,{ flo, flo2 }, 5.5))
>     print(StatTime(1e8,{ ce, ce2 }, 5.5))
>
> Try this
>     print(StatTime(1e8,{ math.floor, flo2 }, 5.5))
>     print(StatTime(1e8,{ math.ceil, ce2 }, 5.5))
>
> and you will see that the bult in functions are indeed faster.
[...]

But I don't think this objection is correct, since I did it on purpose 
to wrap floor and ceil in flo and ce, in order to compare them on par 
with flo2 and ce2, because I wanted to compare them against the idioms
using %, not to an hypothetical reimplementation of ceil and floor using %.

And so it seems that my conclusion is valid: there is no advantage in 
using floor and ceil instead of using % directly, except when 
readability is important (admittedly most of the times when one is using 
Lua and not LuaJIT).

Anyway this is a big gotcha, IMHO: one wouldn't expect that a library 
function would be slower than a do-it-yourself alternative.

But the question about "useless fat" in Lua remains: why then log10 was 
dropped and ceil and floor remain? It's only a matter of statistical 
relevance? (I suppose that most people use floor and ceil much more 
often than log10). I remember Lua team, replying to people complaining 
about the missing log10 that they could roll their own. Why floor and 
ceil are so special?


thanks!
-- Lorenzo


P.S.: has anyone given a look at my benchmarking function? I hope I 
didn't do something too stupid there (especially with the collectgarbage 
sequence etc.)