Re: Dynamic module loading problem

[Philipp Janda <siffiejoe@...>]

Am 19.09.2013 17:10 schröbte Francesco Santini:
> Hello all,

Hi!

> My setup is the following: a program (that is provided to me as-is) is
> dynamically loading a .so library (written by me) that contains the Lua
> VM (linked dynamically, but I also tried statically). My .so file wants
> to load a binary module (which, when loaded from command-line, works fine).
>
> At this point, the "require" fails with the error "unresolved symbol:
> lua_gettop". Other modules give other lua-related unresolved symbols.

Excerpt from `man 3 dlopen`:
External references in the library are resolved using the libraries in 
that library's dependency list and any other libraries previously opened 
with the RTLD_GLOBAL flag. If the executable was linked with the flag 
"-rdynamic" (or, synonymously, "--export-dynamic"), then the global 
symbols in the executable will also be used to resolve references in a 
dynamically loaded library.

So the Lua API is not in the module's dependency list, and also not 
exported by a previously loaded library opened with the RTLD_GLOBAL flag.

> I have to point out that with a previous version of the main program
> this procedure used to work fine.

They probably used RTLD_GLOBAL to load your library before.

> I read that I should enable the option -Wl,-E at linking, which I did
> for my .so file, and in fact my .so file does export all the correct lua
> symbols (as confirmed by nm), but that doesn't help. Is the main program
> supposed to by compiled with -Wl,-E? Provided that I have no control
> over how that program is linked, is there a workaround?

`-Wl,-E` (which is the same as `-Wl,--export-dynamic`, see above) makes 
the executable act as a library opened with RTLD_GLOBAL, this flag 
doesn't have an effect when building libraries.

One obvious solution would be to add the Lua API to the dependency lists 
for all modules (i.e. link all modules with -llua). As long as your 
library and all the modules link to the shared Lua library, you 
shouldn't have any problems with multiple runtimes. Downside is that you 
have to modify all build scripts, and you can't (probably) use those 
modules with the standard lua interpreter.

> I guess linking lua with the module would help, but I would like to
> avoid that if possible.

There is a glibc-specific dlopen-flag (man 3 dlopen):
RTLD_NOLOAD (since glibc 2.2)
    Don't load the library. This can be used to test if the library is 
already resident (dlopen() returns NULL if it is not, or the library's 
handle if it is resident). This flag can also be used to promote the 
flags on a library that is already loaded. For example, a library that 
was previously loaded with RTLD_LOCAL can be reopened with RTLD_NOLOAD | 
RTLD_GLOBAL. This flag is not specified in POSIX.1-2001.

I don't know if this also works from within the dynamically loaded 
library, but you can try (and tell us how it went) ...

> Thanks in advance!
>
> Francesco

Philipp