RE: Reducing memory consumption

["Nick Trout" <nick@...>]

> On Behalf Of Rici Lake
> Sent: Friday, August 27, 2004 8:39 PM
> 
> On 27-Aug-04, at 9:05 PM, Edgar Toernig wrote:
> 
> > /* C side */
> > int
> > gimme_funcs(lua_State *L)
> > {
> >     lua_pushcfunction(L, myFnCallback1);
> >     lua_pushcfunction(L, myFnCallbackN);
> >     return 5;
> > }
> 
> Nice, but it won't work with thousands of functions.
> 
> Here's one that will, provided that an individual script file only
> references, say, 100:
> 
> /* C side */
> 
> #define N_FUNCS 1500
> static const lua_CFunction funcs[N_FUNCS] = {
>    myFnCallback1,
>    myFnCallback2,
>    myFnCallback3,
>    ...
>    myFnCallback1500
> };
> 
> int i;
> 
> /* Fast but maybe not future proof way of creating a vector */
> static const char *makevec = "return function(...) return arg end";
> luaL_loadbuffer(L, makevec, strlen(makevec), "");
> lua_call(L, 0, 1);
> 
> lua_checkstack(L, N_FUNCS);
> for (i = 0, i < N_FUNCS, i++) {
>    lua_pushcfunction(L, funcs[i]);
> }
> lua_call(L, N_FUNCS, 1);
> lua_setglobal(L, "funcs");
> 
> 
> -- Lua side, each script
> local ThisFunctionNameIsVeryDescriptive1,
>        ThisFunctionNameIsVeryDescriptive23,
>        ThisFunctionNameIsVeryDescriptive117,
>        ...
> do
>    local function setvals()
>      ThisFunctionNameIsVeryDescriptive1,
>      ThisFunctionNameIsVeryDescriptive23,
>      ThisFunctionNameIsVeryDescriptive117, ...
>      = funcs[1], funcs[23], funcs[117], ...
>    end
> 
>    setvals()
> end  -- Poof!
> 
> -- rest of script
> 
> and after loading all the scripts, you could do:
> 
> funcs = nil -- Poof, again!
> 
> Personally, I would implement this by using the same name in
> both C and Lua. A simple pattern match against the C files
> could construct the index from function names to index values
> (and also generate the funcs array). You could use a simple
> pattern match against the Lua scripts, too, or you could
> run lparse against them and then iterate over the globals
> that it discovers.
> 
> For maximum poof-ness, you would want to arrange to get
> rid of the C funcs vector as well. Perhaps it could become
> a fixed-length buffer or something :)
> 
> The end result is the creation of an upvalue for each unique
> function in each script, plus a closure for each function
> actually used. But no character strings.

I think this involves some static analysis of the script during
compilation in order to import the script closure so we can call it.
Again, we need the script closures. The method I was suggesting involves
one function closure (for the dispatch) and then a function name to
number substitution in a pre-processing phase (either lexical, regex or
processing Luac -l output). 

The only other way I could come up with of doing this without the
function name for index argument substitution is to have the dispatch
function return a closure which would then be invoked, calling the
function we want. A C or Lua function could be returned. Returning a Lua
function would create garbage (eek) and using a C function would mean 2
function calls for every Lua call (inefficient).

Nick