Re: [ANN] Luabins ? Trivial Lua Binary Serialization Library

[Alexander Gladysh <agladysh@...>]

On Mon, Feb 23, 2009 at 4:18 AM, Luiz Henrique de Figueiredo <lhf@tecgraf.puc-rio.br> wrote:
>> I wrote a small and simple library called "luabins" to serialize and
>> unserialize trivial Lua values (including tables) into arbitrary
>> binary data format.

> My approach to this problem is to use Lua bytecode format instead of
> inventing yet another binary format. My experimental code for that
> approach is available at
>        http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/5.1/save.tar.gz

> See also
>        http://lua-users.org/lists/lua-l/2007-03/msg00546.html
>        http://lua-users.org/lists/lua-l/2007-10/msg00519.html

I've seen your solution. But I have some problems with it:

1. Lua bytecode is unsafe and may crash Lua. Thus text format is to be used.
2. Even text must be verified before fed to loadstring() -- loadstring() is too all-powerful for our purposes.
3. Loadstring on text is slow (compared to specialized binary data loader). 

Or at least so say my silly synthetic benchmarks (see http://github.com/agladysh/luabins/blob/v0.1/etc/benchmark.lua). 

Note also huge penalty on loadstring() in LuaJIT.

Lua
-------------------------------------------------------------------
                name |     rel | abs s / iter = us (1e-6 s) / iter
-------------------------------------------------------------------
        luabins_load |  1.0000 |   6.34 /    1000000 = 6.340000 us
          loadstring |  3.6530 |  23.16 /    1000000 = 23.160000 us

LuaJIT -O
-------------------------------------------------------------------
                name |     rel | abs s / iter = us (1e-6 s) / iter
-------------------------------------------------------------------

        luabins_load |  1.0000 |   5.40 /    1000000 = 5.400000 us
          loadstring | 23.6370 | 127.64 /    1000000 = 127.640000 us

Here are benchmark results on large random dataset (if any one wants to reproduce benchmarks, please say so, I'll publish missing parts):

Lua
-------------------------------------------------------------------
                name |     rel | abs s / iter = us (1e-6 s) / iter
-------------------------------------------------------------------
        luabins_load |  1.0000 |   8.57 /    1000000 = 8.570000 us
         loadstring |  8.8751 |  76.06 /    1000000 = 76.060000 us
 LuaJIT -O
-------------------------------------------------------------------
                name |     rel | abs s / iter = us (1e-6 s) / iter
-------------------------------------------------------------------
        luabins_load |  1.0000 |  10.69 /    1000000 = 10.690000 us
          loadstring | 19.3882 | 207.26 /    1000000 = 207.260000 us

Alexander.