Re: [ANN] lua-MessagePack 0.1.0

[Matthieu Tourne <matthieu.tourne@...>]

Hello!

On Sun, Nov 11, 2012 at 9:56 PM, Alexander Gladysh <agladysh@gmail.com> wrote:

On Sun, Nov 11, 2012 at 12:28 PM, François Perrad
<francois.perrad@gadz.org> wrote:
> I am pleased to announce the first release of lua-MessagePack.
> MessagePack (http://msgpack.org/) is an efficient binary serialization format.

Congratulations on the release!

> lua-MessagePack is not the first implementation of MessagePack.
> It's a pure Lua implementation, without dependency.
> And it's really fast with LuaJIT.

Any benchmarks?

I just ran some tests against the Lua MessagePack libraries I was considering using. 

They are based roughly on this file : https://github.com/catwell/luajit-msgpack-pure/blob/master/tests/bench.lua

Here are my findings :

** https://github.com/fperrad/lua-MessagePack **

$ luajit bench_MessagePack.lua 1000000                     

    empty 0.590000 sec 1694915 times/sec

    iary1 1.230000 sec 813008 times/sec

   iary10 4.750000 sec 210526 times/sec

  iary100 3.530000 sec 28328 times/sec

 iary1000 3.780000 sec 2645 times/sec

iary10000 4.560000 sec 219 times/sec

    dary1 0.040000 sec 500000 times/sec

   dary10 0.070000 sec 142857 times/sec

  dary100 0.050000 sec 19999 times/sec

 dary1000 0.490000 sec 2040 times/sec

     str1 1.460000 sec 684931 times/sec

    str10 1.430000 sec 699300 times/sec

   str100 1.880000 sec 531914 times/sec

  str1000 2.240000 sec 446428 times/sec

 str10000 0.570000 sec 175438 times/sec

 str20000 0.940000 sec 106382 times/sec

 str30000 1.330000 sec 75187 times/sec

 str40000 0.170000 sec 58823 times/sec

 str80000 0.320000 sec 31249 times/sec

** https://github.com/catwell/luajit-msgpack-pure **

luajit bench_msgpack-pure.lua 1000000                   

    empty 0.500000 sec 2000000 times/sec

    iary1 0.950000 sec 1052631 times/sec

   iary10 3.210000 sec 311526 times/sec

  iary100 2.550000 sec 39215 times/sec

 iary1000 9.320000 sec 1072 times/sec

iary10000 12.710000 sec 78 times/sec

    dary1 0.070000 sec 285714 times/sec

   dary10 0.240000 sec 41666 times/sec

  dary100 0.220000 sec 4545 times/sec

 dary1000 2.070000 sec 483 times/sec

     str1 1.030000 sec 970873 times/sec

    str10 1.050000 sec 952380 times/sec

   str100 3.040000 sec 328947 times/sec

  str1000 3.390000 sec 294985 times/sec

 str10000 0.970000 sec 103092 times/sec

 str20000 1.380000 sec 72463 times/sec

 str30000 1.850000 sec 54054 times/sec

 str40000 0.230000 sec 43478 times/sec

 str80000 0.390000 sec 25641 times/sec

** https://github.com/antirez/lua-cmsgpack **

$ luajit bench_cmsgpack.lua 1000000                       

    empty 0.230000 sec 4347826 times/sec

    iary1 0.450000 sec 2222222 times/sec

   iary10 1.850000 sec 540540 times/sec

  iary100 1.240000 sec 80645 times/sec

 iary1000 1.100000 sec 9090 times/sec

iary10000 1.150000 sec 869 times/sec

    dary1 0.010000 sec 1999999 times/sec

   dary10 0.020000 sec 500000 times/sec

  dary100 0.020000 sec 49999 times/sec

 dary1000 0.110000 sec 9090 times/sec

     str1 0.480000 sec 2083333 times/sec

    str10 0.530000 sec 1886792 times/sec

   str100 0.630000 sec 1587301 times/sec

  str1000 0.980000 sec 1020408 times/sec

 str10000 0.420000 sec 238095 times/sec

 str20000 0.780000 sec 128205 times/sec

 str30000 1.220000 sec 81967 times/sec

 str40000 0.150000 sec 66666 times/sec

 str80000 0.310000 sec 32258 times/sec

All those tests were performed with LuaJIT-2.0.0-rc3 on a modern Intel laptop.

The pure C implementation from cmsgpack seems to perform the best, if you don't have the pure Lua constraint.

luajit-msgpack-pure seems on par with lua-MessagePack, except for some bad edge cases (iary10000, dary1000)

Mind you, I have not verified the correctness of unpack(pack(data)), assuming all 3 libraries work as expected.

Feel free to criticize my quick and dirty bench!

Cheers,

Matthieu.