Re: Dump large tables quickly

[Niccolo Medici <niccolomedici@...>]

On 6/30/18, Ravi Joshi <ravi.joshi53@yahoo.com> wrote:
> Hi all,
>
> I am using Lua 5.2. I have large tables (1-dimensional arrays) of size
> 800,000. I want to dump these tables quickly. I found an article on Wiki
> titled "Save Table To File" [1] and used it but found not up to the mark.
> The tables saved using this method, i.e., table.save( table , filename ) are
> shared in my DropBox here [2].
>
> Since my primary concern is speed, hence I am ready to adopt binary file
> serialization if such exists.
>
> References-
> [1] http://lua-users.org/wiki/SaveTableToFile
> [2] https://www.dropbox.com/s/efdfvgy1n4m0zbx/table.dump?dl=0


On my ancient computer, the following code takes 8.5 seconds:

    local t = assert(loadfile('table.dump'))()[1]

    for i = 1, #t do
      print(t[i], ",")
    end

And using this simple-minded C code, it takes 1.3 seconds:

    static int
    l_dump(lua_State * L)
    {
       int len;
       int idx;

       printf("{");
       len = lua_rawlen(L, -1);
       for (idx = 1; idx <= len; idx++) {
         lua_Number n;
         lua_rawgeti(L, -1, idx);
         printf("%g,\n", (double) lua_tonumber(L, -1));
         lua_pop(L, 1);
       }
       printf("}");

       return 0;
    }

That's my first try: I guess this code can be made faster (and you
should make it a luarock, and also make it portable among the Lua
versions). But first search in luarocks.org to see if somebody has
done this already.