Re: module loading in C

[Hans van der Meer <H.vanderMeer@...>]

I received two reactions, thanks.

Luiz Henrique de Figueiredo wrote:
> > > I have module A loaded from a lua program wth require, while >  
> > submodules A1, A2, .. are loaded by A on demand.
> It seems easier to me to add the submodules to package.preload.
I cannot see how, because the loading is dynamic. It is not known  
beforehand what submodules will be present and their number may vary.

David Burgess wrote:
> My bet is that you have a modified Global thread environment. If you  
> have called require() beforehand it is loaded into the master global  
> environment and the _G.package table. When calling your function,  
> are these two references being created in "your" LUA_GLOBALSINDEX or  
> the master one?
Indeed I had in the main module:
lua_newtable(L); /* table for C-function environment */
lua_replace(L, LUA_ENVIRONINDEX); /* set it */
But removing these did not make he problem go away.

I tried various other things like also putting the loaded module table  
inside a "loaded"-table in the LUA_ENVIRONINDEX table. Nothing helped.

Some more experiments were made in the hope they will provide enough  
information for someone to pinpoint the cause of the problem. Putting  
prints in Lua's loadlib.c shows the following sequence of events just  
before a crash (segmentation fault):

(just before this iom-close was called successfully)
>>>> gctm called for 0x107350 <<<<<
>>>> ll_unloadlib LUA_DL_DLOPEN 0x107350 <<<<
TRACE: entering <io_gc> in module ioio
TRACE: io_gc called for readtest.txt.mem
TRACE: entering <close> in module ioio
TRACE: package 'iom' is loaded
TRACE: calling subsystem iom for close
Segmentation fault

0x107350 is the address of the module 'iom' in which the close  
function should be called. As can be seen, by that time the module is  
already unloaded.
All this occurs in the last phase of the program execution, where  
apparently the garbage collector is removing all leftovers. In this  
case for userdata 'readtest.txt.mem' the __gc function is entered and  
the close function in the iom submodule called for it.
The line "TRACE: package 'iom' is loaded" comes from querying the  
LUA_ENVIRONINDEX-table where the iom module table is put into and  
still appears to be at that point.

QUESTION:
I suspect that the cleanup of the garbage collector (in the last phase  
of execution) occurs in the order in which the objects are  
encountered. If so, it is detrimental if the order of unloading is  
critical, as is the case here.
I expected that the parent module ioio (not yet unloaded) and having  
the iom table in its environment, would prevent early unloading even  
in the final cleanup; clearly that is not true. However, I fail to see  
why the iom module could be marked eligible for gc'ing while still  
held by its parent ioio? Is the library perhaps some other object than  
the table granting access to it? I am not versed enough in matters of  
library loading to answer that question.
Is my analysis correct?
Is there a way to solve this?

Thanks in advance for taking your time in replying.
Hans van der Meer



On 2 sep 2008, at 22:46, Hans van der Meer wrote:

> I am wrestling with keeping modules from crashing(disappearing?).
>
> I have module A loaded from a lua program wth require, while  
> submodules A1, A2, .. are loaded by A on demand. The code for the  
> submodule loading:
>
> lua_getglobal(L, subsystem); /* char *subsystem */
> if (lua_isnil(L, -1)) {
>    lua_pop(L, 1); /* pop the nil */
>    lua_getglobal(L, "require");  /* setup call to require */
>    lua_pushstring(L, subsystem); /* argument is name of module to  
> load */
>    if (lua_pcall(L, 1, 1, 0)) luaL_error(L, "require failed");
> }
>
> If I have in the Lua program a statement:
> subsys = require "A1"
> before the programmatic loading will occur all is well, leave it out  
> and the program may crash when trying to enter a function in the  
> submodule A1. This occurs in the endgame of the program execution.  
> This follows from the fact that putting an offending operation  
> within a local do-end block has the garbage collector kick in at an  
> earlier moment.
>
> I tried to fixate the module by putting the table returned by the  
> pcall to require in either LUA_ENVIRONINDEX or LUA_GLOBALSINDEX but  
> neither does work.
> Clearly I am missing an essential point. But what?
> I hope someone can point me in the right direction.