Binary Modules for Lua [was Re: External modules]

["Thatcher Ulrich" <tu@...>]

I took the liberty of putting together the bare bones of a Binary
Module system for Lua 4.0, based on Ignacio's uselib code.  I put up a
page on the wiki, with a source patch and an interpreter binary for
Linux (Win32 to come when I get a chance), and a list of ready-to-use
modules.  (The list of modules consists of one thing at the moment: a
version of luaSDL, which I've made available before in various forms.)

Here's the page:

http://lua-users.org/wiki/LuaBinaryModules

Consider this my initial proposal for a "community standard" binary
module system.  This is One Person's Humble Opinion of what it should
look like.  I'm pretty happy with the general shape of it, however I
probably got some things wrong.  I'd like to work towards some
consensus among users, developers, and the general Lua community, in
order to have a successful community standard.  So, please post your
comments and criticisms.

NOTES:

* The proposal is functional, but not complete.  Mainly it needs
  documentation and perhaps some better organization.  For example,
  some platform-specific advice on where to put binary modules.  Also,
  the source patch needs to include working build scripts for other
  platforms and compilers.

* I made one significant change to Ignacio's uselib code, which was to
  use the keyword "loadmodule" instead of "use" for the function that
  loads a shared library.  I think "use" is too generic a word to use
  for a specialized function such as this.

* This scope of this proposal extends only to *binary* modules written
  in C (or other non-Lua languages).  It is not meant to cover
  conventions for modules and libraries written in Lua.

* I kept Ignacio's original platform-specific library path semantics,
  instead of the stripped-down "loadlib" semantics from the Lua
  authors.  The "loadlib" proposal has a very minimal library loader,
  with no platform-specific path or name-convention knowledge, and
  places those details of library paths and file extensions in a Lua
  code wrapper.  This Lua wrapper would either be associated with each
  potential binary module, or perhaps a generic one could be written
  for each platform.

  This is clever and minimal, but in my opinion, will be less friendly
  to users and module authors.  The rules for loading binary modules
  are a naturally platform-specific thing, and therefore are best
  implemented in C.  Lua does not have a good alternative for C's
  #ifdef, for the kinds of nitty-gritty decisions that need to be
  made.  For example, some platforms which lack dynamic linking may
  prefer to substitute application-specific statically-linked
  initialization code, and implement "loadmodule" accordingly.

  Therefore, I think a generic Lua-based "loadlib" wrapper is
  problematic.  On the other hand, using separate Lua wrappers for
  each module invites confusion.  Paths and naming conventions should
  be determined per-platform, not per-module.  Also, it puts module
  authors in the position of maintaining platform conventions, instead
  of centralizing those worries in the "loadmodule" extension.

  I also think it's nice to be able to distribute a binary module that
  doesn't need a Lua wrapper at all -- only one file, only one path to
  worry about.  The current version of luaSDL falls in this category,
  and I suspect other useful binary modules might also.

  I could be wrong about wanting to avoid a mandatory Lua wrapper; I'd
  like to hear reactions from other library authors.

* I'm DEFINITELY NOT proposing that "loadmodule" be built into the
  standard Lua interpreter.  I think this community standard can
  function just fine as a consensus specification, centralized on some
  pages at lua-users.org, without altering the standard Lua
  distribution in any way.

-- 
Thatcher Ulrich
http://tulrich.com