Re: shared module chain under linux

[Jay Carlson <nop@...>]

On Thu, Feb 23, 2012 at 8:28 PM, Roberto Ierusalimschy
<roberto@inf.puc-rio.br> wrote:
>> I might be talking nonsense here (I'm also dealing with
>> multi-platform dll/so/dylib issues occasionally), but shouldn't
>> Worker.so somehow directly refer, or has implicit dynamic linking to
>> core.so?
>>
>> Wouldn't be worse if use GLOBAL, then you can't have the same symbol
>> exported by two different shared libs (and that's okay for most of
>> the cases, but how would plugins work, usually they have one entry
>> point, and most of the time it's the same symbol name)?
>
> In 5.2, the function package.loadlib has an aditional parameter that
> corresponds to this GLOBAL option. 'require' still links modules with
> private names. If you want to link a C library to use its functions
> directly from another C library, then you should load the library using
> package;loadlib plus that option (instead of using require).

I can't speak with authority on what Windows or OS X best practices
are, but for Linux and all other SVR4 ELF platforms (and some
predecessors) you should always link loadable objects with all of
their dependencies. That is, if you have a module lxp.so, you should
generate it with "gcc -shared lxp.o -o lxp.so -lexpat", which will add
libexpat to the DT_NEEDED list for lxp.so. (In addition, if symbol
versioning is in use in libexpat.so, lxp.so will get references to the
symbol versions it actually uses which is not simple to simulate
otherwise afaik.) This is in how Ubuntu ships:

$ ldd /usr/lib/x86_64-linux-gnu/liblua5.1-expat.so.0
	linux-vdso.so.1 =>  (0x00007fff5a3ff000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f6c138af000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6c13510000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f6c13cfb000)

Dynamic loading of that lxp.so will call in libexpat as needed. This
gets more important as you link in libraries with constructors;
letting ld.so handle the loading of libexpat.so means libexpat.so's
constructors will only be called once. (You can use readelf -d
/lib/whatever | grep NEEDED to see this information directly; I just
use ldd out of habit.)

If you have two lua wrappers which want to intercall, yes, you'll need
something like GLOBAL, but you're better off moving their shared code
into a single library, which both can link with. If you have a lua
wrapper C module which wants to behave differently depending on
whether a symbol is present due to other loads, use a weak symbol
reference. But consider whether you want to be cooperating via the Lua
interface you share instead.

The reason the Lua distribution keeps on getting stuck
finding/explaining all these weirdo dynamic linker/loader options is
that under this rule, lxp.so should also link with -llua5.1. But
/usr/bin/lua is instead serving as an anonymous shared library--one
which cannot be called for by name. If targeting larger apps embedding
lua dynamically, plugins are stuck--should they declare their
dependency on -llua5.1? If they do they can't be used with
/usr/bin/lua. Apache httpd and perl and Python have to deal with
symbol exports from the main program too, but there is not a
libapache....

(At one point I considered declaring all of the liblua.so exports to
be weak; that way it was clear a main program would always override
them, and then loadable modules could always specify -llua5.1.)

By the way, all the shared objects on the system link to their
dependencies, even libm:

$ ldd /lib/x86_64-linux-gnu/libm.so.6
	linux-vdso.so.1 =>  (0x00007fffe2fff000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb926632000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fb926c71000)

The exception is things like perl and Python DSOs, which for similar
reasons cannot specify they want to be linked with -l/usr/bin/python.

Jay