Re: Why does the default cpath not have lib?.so in it?

[David Manura <dm.lua@...>]

On Wed, Sep 14, 2011 at 1:16 PM, Tim Caswell <tim@creationix.com> wrote:
> Are there systems where you have both .dll files and .so files?

In Cygwin, shared libraries are implemented as DLLs (i.e. Windows PE
COFF format).  Conventionally, Cygwin system shared libraries have an
extension ".dll" and prefix "cyg".  Furthermore, they are located in
the /usr/bin directory because default Windows behavior is to search
for DLLs in the PATH.  Import libraries (.dll.a) for those shared
libraries and static libraries (.a) go in /usr/lib as expected for
*nix linking (e.g. gcc -l).  For example, here's what the Lua package
install in Cygwin looks like:

  /usr/lib/liblua.a -> liblua5.1.a
  /usr/lib/liblua.dll.a -> liblua5.1.dll.a
  /usr/lib/liblua5.1.a
  /usr/lib/liblua5.1.dll.a
  /usr/bin/lua.exe
  /usr/bin/luac.exe
  /usr/bin/cyglua-5.1.dll

Now, sometimes you will see a Cygwin DLL that has a ".so" extension.
However, this is mostly just a failure to conform to conventions.
This may result from building some software using a handwritten
Makefile, not developed with strong portability in mind, that
hard-codes a .so extension.  It still works though: Neither Cygwin nor
Windows cares about the DLL extension.

One place this is a little confusing is if you build a vanilla Lua
(lua-5.1.4.tar.gz) on Cygwin.  Cygwin is more *unix than it is
Windows, so you can build it with "make linux" or "make posix".
Unless customized, the Lua 5.1.4 Makefile hard-codes a ".so" (not
".dll") extension in the default package.cpath (i.e. luaconf.h
LUA_CPATH_DEFAULT).  Is this unconventional for Cygwin or for Lua?
Cygwin doesn't really say except by inference to suggest ".dll" .  The
Lua Reference manual doesn't specify except to suggest ".so", and
whatever the "actual behavior" of Lua 5.1.4 may be, it isn't to be
relied on since the Makefile that comes with Lua 5.1.4 isn't designed
to build a shared library version of Lua anyway without some hacking
(required flags to do it in Cygwin are documented separately in [1]).
We've been down this road to uncertainty before (e.g. lua5.1.dll v.s.
lua51.dll anyone?).  The LuaDist distribution of Lua [2], on the other
hand, which intends to follow shared library conventions on each
system, defaults the LUA_CPATH_DEFAULT to use the file extension
reported by the CMake CMAKE_SHARED_MODULE_SUFFIX variable, which under
Cygwin (and Windows) is ".dll".  Cygwin's official Lua distribution
uses ".dll" too.

[1] http://lua-users.org/wiki/BuildingLua
[2] https://github.com/LuaDist/lua/blob/master/CMakeLists.txt

> so I may be way off base

You're not (but if you'd like to be, just join the recent "binaries"
thread ;) ).

On Wed, Sep 14, 2011 at 9:54 AM, Sebastien Lai <237482@googlemail.com> wrote:
> On Wed, Sep 14, 2011 at 8:07 AM, Tim Caswell <tim@creationix.com> wrote:
>> FWIW, the way we solved this in nodeJS is to change the extension to .node
>> and require can only open those and not arbitrary .so files.
> But doesn't that introduce additional ambiguity?
> As far as I know, Lua keeps it that way to remove any possible
> ambiguity. Just by looking at the file extension ".node" it's not
> possible to tell whether it's a DLL, SO, or a simple javascript file.

What do we gain by disambiguating Lua C library names based on their
format (.so for ELF and .dll for PE COFF)?  This still doesn't convey
important compatibility info like 32-bit v.s. 64-bit, Debug v.s.
Release, C library version, CPU architecture, etc.  Neither does Lua
bytecode (.luac), which isn't even in LUA_PATH_DEFAULT.  You need to
examine the header.  What we lose is that we need to worry about using
.so on *nix and .dll on Windows and who-knows-which (probably .dll) on
Cygwin.  So, why bother?  It's common, for example, for ZIP files to
have some other extension (e.g. Lua .rock [3] files).  However, in
comparison, Python C libraries *do* have extensions '.so' in Linux and
'.dll' in Windows and Cygwin.

Also to consider: Lua's packge.path/package.cpath is fairly low level,
and it would help having a standard-ish module (maybe in penlight) to
simplify appending a directory to them while avoiding technicalities
like writing paths twice ("<dir>/?.lua" and "<dir>/?/init.lua" forms),
dealing with extensions (".so" and ".dll"), and doing string
concatenation (";").

[3] http://luarocks.org/en/Types_of_rocks