Re: [Experiment] Module system based on static hashing

[Sean Conner <sean@...>]

It was thus said that the Great Stefan once stated:
> 
> Am 17.04.2020 um 01:17 schrieb Sean Conner:
> > At startup, I load all the luaopen_*() calls
> > into package.preload (these are listed in a luaL_Reg[] arraay) and add a
> > special loader to package.searchers to load the modules in Lua (that are
> > compiled via luac, compressed [1] and stored in the executable; the loader
> > finds the appropriate data and decompresses it when lua_load() is called).
> 
> Did you create it manually or with a script?

  It's part of the Makefile.  The implicite Makefile rule I added can
convert a Lua file into an object, first by running luac over the Lua file,
then using a program over that output to generate a C file that is then
compiled into an object file.

> > Each was run 10,000 times and the runtime (and memory, obtained from
> > lua_gc()) averaged. No special compiler options were used but I think this
> > is enough to give us some ballpark figures.  The base (just state creation
> > and deletion) used 2K and gives us a baseline to compare the rest.  The
> > minmin case (just loading package and leaving the rest to be require()ed)
> > took almost three times as long and amost 3 times the momory.  The min case
> > took almost four times as long, and 3 times the memory.  And the full case
> > ... well ... 9 times longer and 7 times the memory.
> 
> Looks congruent to my results.
> 
> This linear growth with the amount of functionality added is
> the problem. Suddenly the script isn't as small and lightweight anymore.
> 
> And require()ing on a per-function basis isn't a solution, IMO.

  It happens when you effectively are linking at runtime.  A statically
compiled C program can start faster than a dynamically linked one, but takes
more memory.  I added a few more tests.  One does a full initialization of a
Lua state:

	L = luaL_newstate();
	luaL_loadlibs(L);

and then executes a script [1] and get:

[spc]lucy:~/source/lua/spsz>./full-exec 10000
new:     20753.100000
lib:    161631.400000
exec:   170847.200000
close:   41177.600000
memory:     17.000000

  The other one does a minimial initialization of the Lua state:

	L = luaL_newstate();
	luaL_requiref(L,"_G",luaopen_base,1);
	luaL_requiref(L,"package",luaopen_package,1);
	luaL_getsubtable(L, LUA_REGISTRYINDEX, LUA_PRELOAD_TABLE);
	luaL_setfuncs(L,preload,0);
	lua_pop(L,3);

(to get the base functions and the package module), and get this:

[spc]lucy:~/source/lua/spsz>./min-exec 10000
new:     26847.300000
lib:     64153.700000
exec:   210198.200000
close:   19289.000000
memory:     11.000000

  So yes, faster creation time at the expense of runtime.  Again, times are
in nanoseconds on a 32-bit system.

  -spc

[1]	I took the script from your repo to run.  It was slightly modified
to read:

	local math      = require "math"
	local b64       = { encode = function(s) return s end }
	local coroutine = require "coroutine"
	
	local coro = coroutine.create(function()
	    local i = 0
	    while true do
	        i = i + 1
	        coroutine.yield(math.cos(i))
	    end
	end)
	
	local x = 123
	for i = 1, 100 do
	    x = math.sin(x) + select(2, coroutine.resume(coro))
	end
	
	return b64.encode(tostring(x))