Re: [ANN] lua-marshal - fast table serialization

[Richard Hundt <richardhundt@...>]

On 01/03/2011 02:48 PM, Alexander Gladysh wrote:
> I've updated my silly serialization benchmark (attached). Lua-Marshal is
> fast enough to be considered (by me) a viable alternative to my luabins
> for cases when I'd need to serialize cyclic tables and functions. (I
> never really needed to serialize them though.)
>
> BTW, I remember that Fabien wanted to serialize such stuff for Metalua.

Thanks for these. I added pluto to the mix and here's the result:

-------------------------------------------------------------------
                name |     rel | abs s / iter = us (1e-6 s) / iter
-------------------------------------------------------------------
        luabins_load |  1.0000 |   0.42 /     100000 = 4.200000 us
        luabins_save |  1.0952 |   0.46 /     100000 = 4.600000 us
             marshal |  1.1429 |   0.48 /     100000 = 4.800000 us
            unengram |  1.4524 |   0.61 /     100000 = 6.100000 us
           unmarshal |  1.7143 |   0.72 /     100000 = 7.200000 us
          loadstring |  3.7857 |   1.59 /     100000 = 15.900000 us
     pluto_unpersist |  3.9762 |   1.67 /     100000 = 16.700000 us
       pluto_persist |  5.8571 |   2.46 /     100000 = 24.600000 us
              concat |  9.3810 |   3.94 /     100000 = 39.400000 us
              engram | 42.1190 |  17.69 /     100000 = 176.900000 us

So I guess from luabins through marshal to pluto we've got a reasonable 
spectrum from light-weight to heavy duty, with proportional performance.
> Engram is too slow to be considered even marginally useful.
> Serialization to Lua source is much faster (and more secure).

This engram implementation remains a toy until I (or somebody) rewrites 
it in C, but even then, it probably has limited value, considering the 
alternatives.

I thought I'd share it in case somebody wants to have some fun with 
generating standard Lua 5.1 bytecode and they want some code to start 
with (hence my comment "Anyway, just a bit of fun." in my original 
mail). I heartily recommend KHMan's++ "A No-Frills Introduction to Lua 
5.1 VM Instructions" if this sort of thing interests you [1]

Cheers,
Richard

[1] 
http://luaforge.net/docman/view.php/83/98/ANoFrillsIntroToLua51VMInstructions.pdf

> git clone https://github.com/agladysh/luamarca
>
> $ ./run_benchmark.sh benchmark.lua 1e5
>
> lua
> -------------------------------------------------------------------
>                  name |     rel | abs s / iter = us (1e-6 s) / iter
> -------------------------------------------------------------------
>          luabins_load |  1.0000 |   0.57 /     100000 = 5.700000 us
>          luabins_save |  1.1404 |   0.65 /     100000 = 6.500000 us
>               marshal |  1.1754 |   0.67 /     100000 = 6.700000 us
>             unmarshal |  1.7193 |   0.98 /     100000 = 9.800000 us
>              unengram |  2.0000 |   1.14 /     100000 = 11.400000 us
>            loadstring |  3.2807 |   1.87 /     100000 = 18.700000 us
>                concat |  9.1754 |   5.23 /     100000 = 52.300000 us
>                  tstr | 12.2982 |   7.01 /     100000 = 70.100000 us
>            tserialize | 17.0351 |   9.71 /     100000 = 97.100000 us
>                engram | 38.3860 |  21.88 /     100000 = 218.800000 us
>
> The full benchmark below was run with bench.engram/unengram commented out.
>
> $ ./run_benchmark.sh benchmark.lua 1e6
>
> lua
> -------------------------------------------------------------------
>                  name |     rel | abs s / iter = us (1e-6 s) / iter
> -------------------------------------------------------------------
>          luabins_load |  1.0000 |   5.68 /    1000000 = 5.680000 us
>          luabins_save |  1.1479 |   6.52 /    1000000 = 6.520000 us
>               marshal |  1.1954 |   6.79 /    1000000 = 6.790000 us
>             unmarshal |  1.5581 |   8.85 /    1000000 = 8.850000 us
>            loadstring |  3.3151 |  18.83 /    1000000 = 18.830000 us
>                concat |  9.4401 |  53.62 /    1000000 = 53.620000 us
>                  tstr | 12.3644 |  70.23 /    1000000 = 70.230000 us
>            tserialize | 16.6021 |  94.30 /    1000000 = 94.300000 us
>
> Modern LJ2 seems to be too smart for my benchmark (or I did break
> something), so no results for LJ2 this time.
>
> Alexander.
>
> P.S. Disclaimer: read benchmark code, run it on your own box, write your
> own benchmark — this one is a toy. But if you'd see a way to improve
> something, please tell me.