Re: Forcing all Lua symbols to be included on Linux?

[Mike Pall <mikelu-0905@...>]

steve donovan wrote:
> My initial search only revealed that other people have had this
> problem.  My application on Linux links against liblua, and all goes
> well until a script tries to load lfs, and then gives the dreaded
> 'undefined symbol: lua_touserdata', which obviously means that the
> linker is not pulling in everything from liblua.
> 
> Naturally, I could just drop lfs.c into my project, but is there a
> general way of forcing the GCC linker to bring in all the symbols from
> a library? -rdynamic seems rather drastic, since my understanding is
> that this makes _everything_ visible dynamically.

The proper way to do this *is* -rdynamic (aka -Wl,-E). It doesn't
export private or hidden symbols. The Lua core carefully defines
its internal linkage as hidden symbols and only exports the public
C API functions.

On a POSIX/ELF system (Linux, *BSD etc.) you have two choices for
a C application which wants to embed Lua:

1. Link the app against the static library (liblua*.a) and pass -Wl,-E
   to GCC when linking -or-
2. link the app against the dynamic library (liblua*.so).

This requires some discipline in building loadable C modules (such
as lfs.so):

- We want the symbols of the Lua core to be picked up from the
  global symbol pool and we don't want to drag in another copy of
  the Lua VM, so ...
- NEVER link a module against the static library and  ...
- DO NOT link a module against any dynamic library which contains
  or links back to the Lua core. I.e. do not create a backward
  linkage dependency (a package dependency is ok).
- Summary: modules should never have a linker line with -llua,
  -llua51, liblua*.a or similar.

If you do not follow this advice, you'll likely end up with *two*
instances of the Lua VM in memory which don't know of each other.
Apart from the bloat, this will crash sooner or later in obscure
places. Usually in the function newkey(), because the static
symbol dummynode differs for both instances.

You'd also lose the ability to use the same modules with different
Lua VMs (like Lua and LuaJIT). These are compatible at the ABI
level (same interface for lua_*/luaL_* functions), but you can't
use two different VMs on the same internal state (causes a crash).

Most distributions get the linkage of Lua and Lua modules right
nowadays (e.g. recent Debian/Ubuntu). Unfortunately quite a few
external modules ship with bad Makefiles or configure scripts.

So better check your installed modules:

$ ldd /usr/lib/lua/5.1/*.so /usr/local/lib/lua/5.1/*.so 2>/dev/null | grep -c 'liblua.*\.so.*'
0    ... MUST be 0 ...

$ cat /usr/lib/lua/5.1/*.so /usr/local/lib/lua/5.1/*.so 2>/dev/null | grep -ca '\$Lua: Lua'
0    ... MUST be 0 ...


[Note that this discussion does not apply to Windows. It doesn't
have a global symbol pool -- all symbols are tagged with a DLL
name. You always need to link the app and all modules dynamically
against lua51.dll. Whichever Lua VM that contains, depends on
which DLL is picked up when the app is loaded.]

--Mike