Re: [ANN] Lua 5.3.2 (rc1) now available

[Roberto Ierusalimschy <roberto@...>]

> Do you have any benchmarks for the performance improvements resulting
> from the table optimizations in this release?

I gave up benchmarks for these small gains.

Consider this simple program:

  local c = os.clock()
  local a = 0
  for i = 1, 1e8 do a = a + i end
  print(a, os.clock() - c)

It does not allocate memory, does not run GC, does not index tables, it
is quite simple and deterministic. Yet see the results in my machine
(now a Pentium i7):

roberto@arraial:~/prj/lua$ lua temp
5000000050000000	0.820385
roberto@arraial:~/prj/lua$ lua temp
5000000050000000	0.717067
roberto@arraial:~/prj/lua$ lua temp
5000000050000000	0.751981
roberto@arraial:~/prj/lua$ lua temp
5000000050000000	0.875924

('time' gives similar results.)

In a few runs, we get variations greater than 20% in execution
time. This is on par with the gains I expect from those simple
optimizations.  To compare the two Lua versions with some soundness,
I would need statistical analysis to see whether their differences are
statistically significant. (No, it is not enough to run them many times
and compare the mean.)  Otherwise we are comparing noise.  I would also
need to consider whether the differences come from other sources, such
as link order/cache alignments, and whether they persist across
different architectures, compilers, etc.  (Remember that not long ago
I got almost 40% time variations by simply changing the name of a
program.)  [1] and [2] are good papers that discuss these matters.

[1] http://lambda-the-ultimate.org/node/5229
[2] http://www-plan.cs.colorado.edu/klipto/mytkowicz-asplos09.pdf

-- Roberto