Re: Renaming Lua modules (was Re: Modules with standalone main program)

[Sean Conner <sean@...>]

It was thus said that the Great Soni They/Them L. once stated:
> 
> dlopen(NULL, ...) is a thing. Read about it. 

  What you said is ambiguous.  It could mean

	dlopen(NULL), ...) is a thing.  I have read about it.

or

	dlopen(NULL), ...) is a thing.  You should read about it.

  With that said, yes, I am aware of dlopen(NULL,...).  

> It means the symbols need 
> to be exported during linking, even if the linking is static. It just 
> means you have dynamic lookup of "static"-ish symbols. It's complicated.

  Yes.  But the presence of dlopen(NULL) in the code does *NOT*
automatically cause the symbol information to be included in the final
executable.

> If you're turning them straight into a statically linked library, you 
> can still do it but I'm not sure how. (I mean the linker has to find the 
> symbols somewhere, right?)

  Yes.  In the object file itself:

	% cat c1.c
	extern int printf(char const *,...);
	
	int A(int x)
	{
	  printf("%d\n",x);
	  return x;
	}
	% gcc -c c1.c
	% readelf -a c1.o
	  [ output snipped ]

	Symbol table '.symtab' contains 10 entries:
	   Num:    Value  Size Type    Bind   Vis      Ndx Name
	     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND 
	     1: 00000000     0 FILE    LOCAL  DEFAULT  ABS c1.c
	     2: 00000000     0 SECTION LOCAL  DEFAULT    1 
	     3: 00000000     0 SECTION LOCAL  DEFAULT    3 
	     4: 00000000     0 SECTION LOCAL  DEFAULT    4 
	     5: 00000000     0 SECTION LOCAL  DEFAULT    5 
	     6: 00000000     0 SECTION LOCAL  DEFAULT    6 
	     7: 00000000     0 SECTION LOCAL  DEFAULT    7 
	     8: 00000000    30 FUNC    GLOBAL DEFAULT    1 A
	     9: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND printf

	...
	%

> >which involves adding some new entries to the package.searchers array (I
> >update package.loaders but that's because I'm using Lua 5.1 in that
> >particular project).  Another way when statically linking in C based Lua
> >modules is to populate the luaopen_() calls in the package.preload array.
> 
> Assuming you're turning pre-built shared objects into statically linked 
> libraries, this is still trivial.

  I'm not sure how you think this is done, or where.  Yes, it's easy, but it
involves writing C code, not linker magic.

> >>Interesting. So C modules could be renameable if they supported what I
> >>proposed.
> >   Again, how?
> 
> Because they can rely on the passed-in names rather than relying on 
> hardcoded names.

  Again, I think you have a misunderstanding about how this works.  You have
this in Lua:

	local foo = require "foo"

require() will first look in package.loaded for a "foo" entry.  Not finding
it there, it will then call the functions in the package.searchers array. 
The first will attempt to find the "foo" entry in package.preload.  If that
one fails, the next will attempt to look for "foo.lua" (or "foo/init.lua")
using the paths in package.path.  So now we get to the searcher for C based
Lua modules.  It attempts to locate "foo.so" (or "foo.dll" on Windows).

  Okay, it finds a "foo.so".  It loads it via dlopen().  It then needs the
address of the entry point of the module, and it's here were the problem
manifests itself---what is the symbol name?

  It can't be "luaopen".  While what will work on Linux (and probably Mac
OS-X and maybe even Windows, I don't know) there apparently *are* systems
out there with dynamic library support that can't handle the same symbol
from multiple dynamic objects (per an email from Roberto).  It will also
have problems when compiling the modules statically (it will require a
source change).

  The solution Roberto came up with is, look for "luaopen_foo"---in other
words, append the module name to the end of "luaopen_" (and change any "."
in the module name to "_") and look for *that* symbol.  If I simply change
the name of "foo.so" to "bar.so" and try to do:

	local bar = require "bar"

it will fail, because "bar.so" does not contain the symbol "luaopen_bar". 
Renaming the dynamic object does *NOT* change any of the symbols defined
inside the dynamic object, which is why I keep asking you "how".  If the
entry point isn't found, Lua can't even pass in the module name and file to
it.  

  Now, Lua 5.3 *DOES* contain package.loadlib(), which is a wrapper around
dlopen()/dlsym().  So keeping in mind the limitations (there must be a
unique symbol identifying the entry point of a C module), if you can come up
with a way to rename a C-based Lua module and still find that unique symbol
*WITHOUT RECOMPILING THE MODULE*, then by all means, go ahead.  I'd love to
see the solution.

  -spc