RE: Hooks for I/O, memory, etc?

["Joshua Jensen" <jjensen@...>]

> Lua already uses luaM_realloc for all its memory allocation. 
> You can build Lua to use a different function by #defining  
> l_realloc. But you can also provide your own luaM_realloc 
> function and the linker will use your version instead of 
> Lua's. Isn't that flexible enough? Does it work with dll/so? --lhf

This doesn't work in a DLL and is purely dependent on link order as a
.LIB under Windows.

I posted a code solution yesterday that fully brings this up against Lua
4.1 work4.  I don't want to say my solution is right, but it does work
and fixes every problem described thus far on the list regarding memory
management.

Let me address a comment from Roberto in another mail:

>If we call it before lua_open, where would we store such
>value? (Lua uses no global/static variables. Everything
>in Lua is stored in a lua_State, which is created by
>lua_open.)

You store the default callback for a new lua_open() in a global/static
variable.  There is no reason why you wouldn't want to do this.  In the
solution I posted, the act of calling lua_open() copies the static
variable contents containing the callback into the global_State for Lua.
Nobody knows those static variables exist.  They're small.  Nobody will
complain about 8 bytes of space missing (and only 4 when you remove the
free() like you said you were going to).  It's efficient, make sense,
and doesn't make the semantics of lua_open() ugly.  Nobody has to even
use the callbacks, as the reasonable defaults are Lua's defaults.

For example:

lua_ReallocFunction oldReallocFunc;
lua_FreeFunction oldFreeFunc;
void* oldData;
lua_getdefaultmemoryfunctions(&oldReallocFunc, &oldFreeFunc, &oldData);
lua_setdefaultmemoryfunctions(luaHelper_ReallocFunction,
luaHelper_FreeFunction, NULL);
lua_State* state = lua_open();
lua_setdefaultmemoryfunctions(oldReallocFunc, oldFreeFunc, oldData);

>What about varargs, with name-value pairs?
>  lua_open("realloc", &my_realloc, NULL)

Ewww...

There are some other reasons to use some static variables to store
information pre-lua_open().  In Amped, we could NOT have fragmented
memory.  That meant that I figured out the maximum size of certain Lua
structures and had several pre-lua_open() calls to address memory
concerns:

For example (from Lua 4.1 Alpha, some of which doesn't apply anymore):

* lua_setminimumstringtablesize(int numstrings) will ensure the global
string table is always of the minimum size specified by numstrings. When
garbage collection occurs, the string table will never shrink below
numstrings. An application can determine the maximum number of strings
it will use and ensure the space is reserved in advance, so as to avoid
fragmentation when the string table is resized.

* lua_setdefaulttagtablesize(int numtags) will create a hash table of
numtags size for the tag table.

* lua_setminimumglobaltablesize(int numentries) ensures the globals
table is capable of at least holding numentries elements without
resizing to avoid fragmenting the heap.

* lua_setminimumauxspace(int size) creates a minimum auxiliary space
buffer of size bytes. Auxiliary space is automatically allocated with
the allocation flag LUA_ALLOC_TEMP. An application might use this to put
auxiliary space at the top of a heap or in some other location, so later
freeing of it doesn't cause fragmentation issues.

It's okay to have a bit of global storage for Lua start-up
functionality.  Or throw it in a structure you pass in, for all I care.
By having it global, all I need to do is change the values I want
changed.  For the structure, I have to understand how to fill in all the
details.  I prefer the global approach, but hey, it's up to you.

Thanks,
Josh