Re: [ANN] lua-MessagePack 0.1.0

[François Perrad <francois.perrad@...>]

2012/11/12 Matthieu Tourne <matthieu.tourne@gmail.com>:
> Hello!
>
> On Sun, Nov 11, 2012 at 9:56 PM, Alexander Gladysh <agladysh@gmail.com>
> wrote:
>>
>> On Sun, Nov 11, 2012 at 12:28 PM, François Perrad
>> <francois.perrad@gadz.org> wrote:
>> > I am pleased to announce the first release of lua-MessagePack.
>> > MessagePack (http://msgpack.org/) is an efficient binary serialization
>> > format.
>>
>> Congratulations on the release!
>>
>> > lua-MessagePack is not the first implementation of MessagePack.
>> > It's a pure Lua implementation, without dependency.
>> > And it's really fast with LuaJIT.
>>
>> Any benchmarks?
>
>
> I just ran some tests against the Lua MessagePack libraries I was
> considering using.
>
> They are based roughly on this file :
> https://github.com/catwell/luajit-msgpack-pure/blob/master/tests/bench.lua
>
> Here are my findings :
>
> ** https://github.com/fperrad/lua-MessagePack **
>
> $ luajit bench_MessagePack.lua 1000000
>     empty 0.590000 sec 1694915 times/sec
>     iary1 1.230000 sec 813008 times/sec
>    iary10 4.750000 sec 210526 times/sec
>   iary100 3.530000 sec 28328 times/sec
>  iary1000 3.780000 sec 2645 times/sec
> iary10000 4.560000 sec 219 times/sec
>     dary1 0.040000 sec 500000 times/sec
>    dary10 0.070000 sec 142857 times/sec
>   dary100 0.050000 sec 19999 times/sec
>  dary1000 0.490000 sec 2040 times/sec
>      str1 1.460000 sec 684931 times/sec
>     str10 1.430000 sec 699300 times/sec
>    str100 1.880000 sec 531914 times/sec
>   str1000 2.240000 sec 446428 times/sec
>  str10000 0.570000 sec 175438 times/sec
>  str20000 0.940000 sec 106382 times/sec
>  str30000 1.330000 sec 75187 times/sec
>  str40000 0.170000 sec 58823 times/sec
>  str80000 0.320000 sec 31249 times/sec
>
>
> ** https://github.com/catwell/luajit-msgpack-pure **
>
> luajit bench_msgpack-pure.lua 1000000
>     empty 0.500000 sec 2000000 times/sec
>     iary1 0.950000 sec 1052631 times/sec
>    iary10 3.210000 sec 311526 times/sec
>   iary100 2.550000 sec 39215 times/sec
>  iary1000 9.320000 sec 1072 times/sec
> iary10000 12.710000 sec 78 times/sec
>     dary1 0.070000 sec 285714 times/sec
>    dary10 0.240000 sec 41666 times/sec
>   dary100 0.220000 sec 4545 times/sec
>  dary1000 2.070000 sec 483 times/sec
>      str1 1.030000 sec 970873 times/sec
>     str10 1.050000 sec 952380 times/sec
>    str100 3.040000 sec 328947 times/sec
>   str1000 3.390000 sec 294985 times/sec
>  str10000 0.970000 sec 103092 times/sec
>  str20000 1.380000 sec 72463 times/sec
>  str30000 1.850000 sec 54054 times/sec
>  str40000 0.230000 sec 43478 times/sec
>  str80000 0.390000 sec 25641 times/sec
>
>
> ** https://github.com/antirez/lua-cmsgpack **
>
> $ luajit bench_cmsgpack.lua 1000000
>     empty 0.230000 sec 4347826 times/sec
>     iary1 0.450000 sec 2222222 times/sec
>    iary10 1.850000 sec 540540 times/sec
>   iary100 1.240000 sec 80645 times/sec
>  iary1000 1.100000 sec 9090 times/sec
> iary10000 1.150000 sec 869 times/sec
>     dary1 0.010000 sec 1999999 times/sec
>    dary10 0.020000 sec 500000 times/sec
>   dary100 0.020000 sec 49999 times/sec
>  dary1000 0.110000 sec 9090 times/sec
>      str1 0.480000 sec 2083333 times/sec
>     str10 0.530000 sec 1886792 times/sec
>    str100 0.630000 sec 1587301 times/sec
>   str1000 0.980000 sec 1020408 times/sec
>  str10000 0.420000 sec 238095 times/sec
>  str20000 0.780000 sec 128205 times/sec
>  str30000 1.220000 sec 81967 times/sec
>  str40000 0.150000 sec 66666 times/sec
>  str80000 0.310000 sec 32258 times/sec
>
>
> All those tests were performed with LuaJIT-2.0.0-rc3 on a modern Intel
> laptop.
>
> The pure C implementation from cmsgpack seems to perform the best, if you
> don't have the pure Lua constraint.
> luajit-msgpack-pure seems on par with lua-MessagePack, except for some bad
> edge cases (iary10000, dary1000)
>

I did some experiments with FFI (see the branch 'ffi'
http://github.com/fperrad/lua-MessagePack/tree/ffi),
but on my hardware, I don't see improvement.

François

> Mind you, I have not verified the correctness of unpack(pack(data)),
> assuming all 3 libraries work as expected.
> Feel free to criticize my quick and dirty bench!
>
> Cheers,
> Matthieu.