Re: The Lua interpreter and large scripts

[Sean Conner <sean@...>]

It was thus said that the Great Mike Pall once stated:
> Luiz Henrique de Figueiredo wrote:
> > > The "-i" option loads readline which as a side-effect initializes
> > > the NLS locale for the current process from the environment variables.
> > 
> > If available, readline is compiled into the interpreter. You don't need
> > to use -i to activate it.  Do you mean that when you do use interactive
> > input then a function from the readline library gets called and only then
> > the OS loads the readline library? And so readline is not loaded at all
> > when running the interpreter in batch?
> 
> I should have been more precise. The shared library is always
> loaded (which is a waste without -i). The first time a readline
> function is called (only with -i), it's initialized and it sets
> the locale.

  The "-i" option is used to load a script and continue into interactive
mode.  If you just run "lua" you still end up in interactive mode.  

  When I run "strace -e open lua" I get the following:

open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/tls/libm.so.6", O_RDONLY)    = 3
open("/lib/libdl.so.2", O_RDONLY)       = 3
open("/usr/lib/libreadline.so.4", O_RDONLY) = 3
open("/usr/lib/libhistory.so.4", O_RDONLY) = 3
open("/usr/lib/libncurses.so.5", O_RDONLY) = 3
open("/lib/tls/libc.so.6", O_RDONLY)    = 3
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
open("/usr/share/terminfo/x/xterm", O_RDONLY) = 3
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
open("/etc/inputrc", O_RDONLY)          = 3
open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3
> 

  That final ">" is the Lua prompt.  Now, let's load a script with "strace
-e open lua -i default.lua" ("default.lua" being the 80M script---this takes
only 15 seconds):

open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/tls/libm.so.6", O_RDONLY)    = 3
open("/lib/libdl.so.2", O_RDONLY)       = 3
open("/usr/lib/libreadline.so.4", O_RDONLY) = 3
open("/usr/lib/libhistory.so.4", O_RDONLY) = 3
open("/usr/lib/libncurses.so.5", O_RDONLY) = 3
open("/lib/tls/libc.so.6", O_RDONLY)    = 3
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
open("default.lua", O_RDONLY)           = 3
open("/usr/share/terminfo/x/xterm", O_RDONLY) = 3
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
open("/etc/inputrc", O_RDONLY)          = 3
open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3
> 

  Hmm ... curious.  The script is loaded first, then the locale stuff.  What
has me really bugged is when I do this:

[spc]lucy:/tmp/lua>lua 
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> dofile("show.lua")	--NOTE!  I'm typing this
> dofile("default.lua") --this too!
> 

it only takes less than 15 seconds.  Yet doing:

[spc]lucy:/tmp/lua>lua -i show.lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> dofile("default.lua")	--NOTE!  I'm typing this
>

takes six minutes.  Other than the slight differences in how the script is
loaded via "-i" on the command line and dofile() from within the
interpreter, I don't why one way should take 15 seconds and the other way 6
minutes.

  So I decided to profile Lua under the various conditions.  First, "lua -i
show.lua / dofile('default.lua')":

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
 97.71    314.31   314.31  2888775     0.00     0.00  luaS_newlstr
  0.34    315.40     1.09  8932847     0.00     0.00  luaH_getstr
  0.30    316.37     0.97  2154477     0.00     0.00  read_string
  0.27    317.23     0.85 10442038     0.00     0.00  llex
  0.13    317.63     0.41 67142148     0.00     0.00  save
  0.08    317.89     0.26  3264403     0.00     0.00  addk
  0.07    318.10     0.21  3891495     0.00     0.00  newkey
	[ snip ]

Doing any other combination ("lua -i default.lua / dofile('show.lua')" or
"lua / dofile('show.lua') dofile('default.lua')" or "lua /
dofile('default.lua') dofile('show.lua')" results in the following profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
 17.75      1.04     1.04  2154477     0.00     0.00  read_string
  9.47      1.59     0.56 10442038     0.00     0.00  llex
  9.39      2.15     0.55  2888775     0.00     0.00  luaS_newlstr
  7.34      2.58     0.43  8932844     0.00     0.00  luaH_getstr
  7.00      2.98     0.41 67142145     0.00     0.00  save
  3.24      3.17     0.19        4     0.05     0.22  luaV_execute
  2.82      3.34     0.17   704940     0.00     0.00  setnodevector
  2.73      3.50     0.16  5422300     0.00     0.00  mainposition
  2.30      3.63     0.14  3891495     0.00     0.00  newkey
	[ snip ]

And yes, all three of those return *identical* profiling results.  And I can
verify that in all four cases, the entirety of the 80m script is loaded.  

  -spc (Curiouser and curiouser ... )