lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

Thanks, Niccolo Medici. It was wonderful to hear your suggestion.

I extended your suggestion and made a sample Lua C binding. Below is the Lua code-

my_table = {p = {11, 22, 33, 44}, q = {0.12, 0.23, 0.34, 0.45, 0.56}}
require "savetable"

Please note that in the above code for debugging purpose, I have defined two very small 1-dimensional arrays. However, in reality, the table contains two 1-dimensional arrays in which each array contains approximately 800,000 elements.

I am not putting the C code using Lua C bindings in this message since it can take many lines here. Instead, please check following Pastebin links to get the code-

(1) savetable.c:
(2) wrapper.lua:

I am looking for suggestions, to make the table serialization much faster. I am using Lua 5.1 (sorry, I said 5.2 previously by mistake) on 64 bit Ubuntu PC. 

PS: I am new to Lua community, I couldn't find much help on the Do you know any such library?

Thanks again


On Sunday, 1 July, 2018, 4:49:01 PM GMT+9, Niccolo Medici <> wrote: 

On 6/30/18, Ravi Joshi <> wrote:

> Hi all,
> I am using Lua 5.2. I have large tables (1-dimensional arrays) of size
> 800,000. I want to dump these tables quickly. I found an article on Wiki
> titled "Save Table To File" [1] and used it but found not up to the mark.
> The tables saved using this method, i.e., table , filename ) are
> shared in my DropBox here [2].
> Since my primary concern is speed, hence I am ready to adopt binary file
> serialization if such exists.
> References-
> [1]
> [2]

On my ancient computer, the following code takes 8.5 seconds:

    local t = assert(loadfile('table.dump'))()[1]

    for i = 1, #t do
      print(t[i], ",")

And using this simple-minded C code, it takes 1.3 seconds:

    static int
    l_dump(lua_State * L)
      int len;
      int idx;

      len = lua_rawlen(L, -1);
      for (idx = 1; idx <= len; idx++) {
        lua_Number n;
        lua_rawgeti(L, -1, idx);
        printf("%g,\n", (double) lua_tonumber(L, -1));
        lua_pop(L, 1);

      return 0;

That's my first try: I guess this code can be made faster (and you
should make it a luarock, and also make it portable among the Lua
versions). But first search in to see if somebody has
done this already.