Re: A major problem with the Lua ecosystem

[Sean Conner <sean@...>]

It was thus said that the Great Dibyendu Majumdar once stated:
> On 1 February 2018 at 23:22, Sean Conner <sean@conman.org> wrote:
> 
> >> Maybe there is one problem - the require function also loads shared
> >> libraries. Having these in multiple locations could be a problem as
> >> these need to be in the path.
> >
> >   It depends upon what you mean.  For example, my syslog module [1] is a
> > C-based module.  When you do:
> >
> >         local syslog = require "org.conman.syslog"
> >
> > Lua will replace the dots with the file separator (in my case '/'):
> >
> >         org/conman/syslog
> >
> 
> Fine, but I meant that shared libraries must be on the system path for
> them to resolve each other hen you have one library depending on
> another. So putting them in various subfolders is not going to work
> unless you also set PATH or LD_LIBRARY_PATH. Maybe the solution is
> that the name by which a module is known should be distinct from the
> path from where it is loaded - or at least user should be able to set
> the desired name,

  Just to make sure we're on the same page, here's what happens when 

	tcc = require "org.conman.tcc"

is loaded (tcc is a C-based Lua module that wraps TCC, a library based C
compiler).  Lua transforms the name of the requested module and searches
package.cpath for the appropriate shared object.  On my system, it finds

	/usr/local/lib/lua/5.1/org/conman/tcc.so

  That path is then passed to dlopen().  dlopen() will then load the
file directly, since the passed in path contains a '/' (and dlopen() will
skip searching for it).  As it's opening the shared object, it checks to see
what other shared objects is needs, and again, in this case:

        libtcc.so
        libc.so.6
        /lib/ld-linux.so.2

  The first two contain no '/', so a search for the shared objects is done,
and results in:

        /usr/local/lib/libtcc.so
        /lib/tls/libc.so.6

(remember---this is all being done in dlopen(), not the Lua require()
function) so those two object are loaded in based on a search.  The library
"/lib/ld-linux.so.2" is loaded directly because it contains a '/' so no
search is neccessary for that shared object.  Once all these are loaded,
dlopen() then returns a valid reference to the shared object tcc.so.

  $LD_LIBRARY_PATH can be used to override the defaults.  For example, I can
place libtcc.so (the C compiler library) into another directory and point to
it via $LD_LIBRAY_PATH:

	libtcc.so => /tmp/foo/libtcc.so (0x008cf000)
        libc.so.6 => /lib/tls/libc.so.6 (0x003fa000)
        /lib/ld-linux.so.2 (0x00b76000)

  Lua modules written in Lua can be renamed easily---just change the name of
the file [1].  Modules written in C can't be renamed without a code change
due to possible linking restrictions [2]---Lua looks for
"luaopen_modulename" when loading a C-based module.

  -spc

[1]	This is for Lua 5.2+.  For Lua 5.1 you can only do this if the Lua
	module in question has the line:

		module(...)

	Otherwise, you'll need to change that line to reflect the new name.

[2]	Names!  I want names of systems that can't load two shared objects
	with the same exported function name!  That way, I know which
	systems to avoid as they're obviously broken!