Re: [ANN] LuaCSV

[Thomas Lauer <thomas.lauer@...>]

steve donovan <steve.j.donovan@gmail.com> wrote:

> On Mon, Aug 3, 2009 at 2:43 PM, Thomas Lauer<thomas.lauer@virgin.net> wrote:
> > I have not yet looked into your code but one question springs to mind
> > immediately: have you done some benchmarks (informal or otherwise), eg
> > with a pure-Lua solution that works with LPEG or some other
> > pattern-matching algorithms? I am confident that a low-level interface
> > would be faster than pure Lua... the question is how much faster?
> 
> The results were rather interesting:
> 
> I compared against the pure Lua solution given in
> http://lua-users.org/wiki/CsvUtils. My first result showed them
> basically taking the same, until I realized that by default I'm doing
> the floating-point conversion.
> 
> The target was a 20meg csv file, randomly generated rows of ten numbers.
> 
> (A) No floating-point conversion:
>  lua: 1.94
>  csv: 0.63
> 
> (B) Floating point conversion:
>  lua: 4.22
>  csv: 1.80
> 
> (This just added up all the numbers)
> 
> So the conversions are expensive; if you need them, it's cheaper to do
> them in C.

Interesting indeed. It seems a carefully optimised LPEG solution is not
too far away from the C code. If one needs raw speed then C is obviously
the way to go, but this looks very promising.

It'll take me some time to have a closer look into this (also
experimenting with some other pattern-matching approaches) but if I find
something interesting I will report back.

-- 
cheers  thomasl