Re: Load large amount of data fast

[Enrico Colombini <erix@...>]

On 17/10/2010 2.57, Alexander Gladysh wrote:
> I've got a large file  (3M entries, 250 MB) with data.
> Each entry is one line with a small Lua table:

This may may be completely off the mark, but just in case...
As you noticed), it sounds like swap thrashing; have you kept an eye on 
memory allocation?

I'd try loading a group of (say 1000) lines at a time and forcing a gc 
after each group to prevent allocation bloating.

--
  Enrico