Re: The Lua interpreter and large scripts

[Sean Conner <sean@...>]

It was thus said that the Great Mike Pall once stated:
> Sean Conner wrote:
> >  97.71    314.31   314.31  2888775     0.00     0.00  luaS_newlstr
> >   9.39      2.15     0.55  2888775     0.00     0.00  luaS_newlstr
> 
> Umm, can you get a line-by-line profile? If not, then maybe
> compile lstring.c with -O2 -fno-inline and see what pops up.
> 
> One explanation for this difference would be excessive string hash
> collisions. But I don't know how that could be triggered by the
> interactive mode (well, it creates a few strings).
> 
> You may want to measure the length of the chain in luaS_newlstr
> and print a message if it exceeds (say) 50. The chain length
> should be somwhere between 1 and 3 on average.
> 
> You could also trace the calls to luaS_resize. The string table
> usually grows and only occasionally shrinks. But there shouldn't
> be continuous resizing.

  Hmm ... interesting.  In the six minute run, luaS_resize is called 469488
times, but in the other three cases, it's only called 15 times.  And in
looking over the profiler output, there are a lot of similarities, but some
very curious differences as well.  

Six minute run:

-----------------------------------------------
                0.00    0.00       1/469488      f_luaopen [136]
                0.14    0.00  469487/469488      newlstr [53]
[62]     0.0    0.14    0.00  469488         luaS_resize [62]
                0.00    0.00      12/2101456     luaM_realloc_ [76]
-----------------------------------------------

15 second run:

-----------------------------------------------
                0.01    0.00       1/15          f_luaopen [107]
                0.08    0.00      14/15          newlstr [49]
[56]     1.5    0.09    0.00      15         luaS_resize [56]
                0.00    0.00      30/2101448     luaM_realloc_ [86]
-----------------------------------------------

  -spc (Perhaps some pathological case I'm hitting?)