lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


It was thus said that the Great Geoff Leyland once stated:
> On 11/04/2014, at 11:14 am, Sean Conner <sean@conman.org> wrote:
> 
> > It was thus said that the Great Geoff Leyland once stated:
> > 
> >> I’ve just had a “real world” situation where reducing temporary table
> >> creation halved the time it took to perform a computation.  Granted, the
> >> original code was probably a bit wasteful with them, and algorithmic
> >> improvements made a bigger difference, but temporary tables were a
> >> measurable factor.
> 
> >  I'd be interested to know of the CPU, CPU speed
> 
> $ uname -v
> Darwin Kernel Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64
> $ sysctl -n machdep.cpu.brand_string
> Intel(R) Core(TM) i7-2675QM CPU @ 2.20GHz
> 
> >  and type (general terms) of computation
> 
> Approximate phrase search using dynamic programs for approximate phrase
> and word matching[1].  For each invocation of the approximate match both
> the phrase matcher and the word matcher allocated tables to hold the
> stages and states of their relevant DPs.  Now I keep a single “static”
> table around and reuse it on each invocation.
> 
> It’s relevant that I run the same tool on more limited hardware where I
> was running out of memory, so I called collectgarbage once per query. 
> Getting memory use to the point that it wasn’t a problem meant I could
> stop doing that, and that also contributed to time savings.
> 
> > Also, the actual difference.
> 
> Original run time was about a minute for a test data set, and “luajit
> -jp=v…” showed about half the time was in the GC.  Cleaning up table
> allocations roughly halved the run times and reduced GC time to about 5%. 
> Fixing a one-line thinko and tuning a parameter reduced the number of
> matches necessary to the point that it now takes a couple of seconds.[2]

  Thanks.  Wow.  Okay.  That is quite significant.  

  I also like that you profiled before changing the code.  

  -spc 

> [1] I know there are alternative methods for this.
> [2] I don’t think this is a case of me making the code fast, more that it’s less slow.