Re: Unloading a C library cleanly

Hi,

Thank you so much for taking the time to write a detailed reply. I didn't realize that the C registry is accessible from Lua and so I wrote a patch for loadlib.c with the C code given later.

Now that I see from your code that this can be done from Lua itself, I'll do that. The other way you suggested using upvalues definitely looks clever, I'll see if that works for my case :)

It does make me nervous that everyone is a bit quiet about unloading libraries. Guess I'll just risk it and hope for the best :D !

C Code:

static int removePathFromInternalCLIBSList(lua_State *L, void *plib) {
int indexToRemove = 0;
lua_rawgetp(L, LUA_REGISTRYINDEX, &CLIBS);
lua_pushnil(L); /* first key */
while (lua_next(L, -2) != 0) {
/* uses 'key' (at index -2) and 'value' (at index -1) */
/*sprintf(buffer + strlen(buffer), "%s - %s\n",
lua_typename(L, lua_type(L, -2)),
lua_typename(L, lua_type(L, -1)));*/

if (lua_type(L, -2) == LUA_TNUMBER) {
//sprintf(buffer + strlen(buffer), "NUMBER: %d -> userdata\n\n", (int)lua_tonumber(L, -2)/*, (int)lua_tonumber(L, -1)*/);
void *plibVal = lua_touserdata(L, -1);
if (plib == plibVal) {
indexToRemove = (int)lua_tonumber(L, -2);
lua_pop(L, 2);
break;
}
}

/* removes 'value'; keeps 'key' for next iteration */
lua_pop(L, 1);
}

// Now the entry at indexToRemove must be removed from the array portion of CLIBS
if (indexToRemove > 0) {
lua_getglobal(L, "table");
lua_getfield(L, -1, "remove");
lua_replace(L, -2); // remove _G.table
lua_rawgetp(L, LUA_REGISTRYINDEX, &CLIBS);
lua_pushinteger(L, indexToRemove);
lua_call(L, 2, 0);
}
else {
return -1;
}
return 0;
}

/*
** registry.CLIBS[path] = plib -- for queries
** registry.CLIBS[#CLIBS + 1] = plib -- also keep a list of all libraries
*/
static int removeCLIBPathFromRegistry(lua_State *L) {
const char *path = luaL_checkstring(L, 1);
void *plib = checkclib(L, path);

// Try clearing the index of the C library from the array part of CLIBS
int errorCode = removePathFromInternalCLIBSList(L, plib);
if (errorCode < 0) {
return luaL_error(L, "package.removeCLIBPathFromRegistry: Unable to locate index of C library in registry.CLIBS: %s", path);
}

lua_rawgetp(L, LUA_REGISTRYINDEX, &CLIBS);
lua_pushnil(L);
lua_setfield(L, -2, path); /* CLIBS[path] = nil */
lua_pop(L, 1); /* pop CLIBS table */
return 0;
}

Thanks again!

Abhijit

On Fri, Dec 6, 2019 at 3:35 AM nobody <nobody+lua-list@afra-berlin.de> wrote:

On 03/12/2019 12.57, Abhijit Nandy wrote:
> /*
> ** return registry.CLIBS[path]
> */
> static void *checkclib (lua_State *L, const char *path) {
> void *plib;
> lua_rawgetp(L, LUA_REGISTRYINDEX, &CLIBS);
> lua_getfield(L, -1, path);
> plib = lua_touserdata(L, -1); /* plib = CLIBS[path] */
> lua_pop(L, 2); /* pop CLIBS table and 'plib' */
> return plib;
> }
>
> So how can I safely clear the LUA_REGISTRYINDEX ( registry.CLIBS[path]
> specifically), so that plib is set to NULL in the function and the DLL
> is again loaded in lsys_load() and the open function in it is called?

Not sure about this… I don't think there's an official way. As no one
else replied to this so far, what could you do to manually achieve this?
(Not sure if I'm missing anything.)

Beyond the loaded library in `package.loaded`, the C library loading
keeps a table behind a userdata/pointer in the registry. (If you search
for that userdata key very early / before loading other libraries, it
should be the only one in the registry. So on the Lua side, you could go

local clibs
for k, v in pairs( debug.getregistry( ) ) do
if type( k ) == "userdata" and type( v ) == "table" then
clibs = v ; break
end
end

and from C you'd use LUA_REGISTRYINDEX for the same purpose.)

Now this table maps both the path to the library and also keeps an array
so that it can unload them in reverse order of loading. It has a __gc
metamethod that closes all C libraries, but that's not useful to unload
single libraries. If (as you say) you're closing the library on your
own, (I think) you only need to

function removelib( path )
local ptr = clibs[path]
clibs[path] = nil
for i, v in ipairs( clibs ) do
-- shift down rest of array
if v == ptr then table.remove( clibs, i ) ; break end
end
end

and then it's gone from there. I've only skimmed the other loadlib
code, so not sure if I missed anything not related to this table.

> Also is there any other cleanup I need to do when unloading the C
> library, so it works exactly as if it has been loaded the first time :)

I haven't a clue…

-----

If you know what OSen you want to support, another approach would be to
use your own library opening routines to load the library. (`require` &
friends are intended for loading stable libraries once and for all
(caching them etc.), whereas you seem to be doing dynamic code
generation or something like that.) That'd do pretty much what openlib
does – open the library, search for lua_openFOO, return that (or call it
directly w/ some arguments & return the result).

My approach would be to push the library reference as a full userdata
with a __gc metamethod that closes it. You'd slightly modify the
lua_openFOO protocol for this: When opening a library, you push the
lua_openFOO function, then the library userdata, then call w/ 1
argument. In lua_openFOO, you add an upvalue to all functions (push the
library reference before calling luaL_setfuncs and specify 1 upvalue for
all (or one more if you already have upvalues).)

Now every function from that library has a reference to the library.
Once you remove the last function, all of them will be gone which means
Lua will collect the upvalue, which means the library will be closed.
Further, there's no caching of the library name / library mapping, so
you (probably) can immediately reuse the name even while the old one
wasn't unloaded yet. (Not sure about dynamic library specifics for
different platforms… I *think* usually the filename only matters for
searching the library to load.)

(Currently in a hurry, hope this is somewhat comprehensible…)

-- nobody