package proposal

[Roberto Ierusalimschy <roberto@...>]

This is a proposal for a more standard package usage in Lua.  It
includes some changes in the `require' function, some changes in the
`luaL_openlib' function, and a new function `module'. We already got
previous comments about this proposal from Diego Nehab (LuaSocket),
Andre Carregal (Kepler) and Tomas Gorham (Kepler).

The following is a description of the new system.  At the end of this
text is an implementation for the functions `require' and `module' in
Lua, which serves as a more precise specification.


* A *package* is a collection of modules distributed together.
Each *module* lives in single file, which contains a single module.
(Of course, a package may comprise a single module.)

* Modules can be loaded in (at least) three forms:
pre-loaded (that is, the module is loaded in the Lua state
during C initialization code, like the standard libraries);
pre-linked (the module code is linked with the Lua executable,
but must be loaded explicitly);
and dynamically (the module code is found through a search in
some path, and then linked and loaded).


* Given a statement <<require"mod">>,
require first checks whether it is already loaded,
looking for a field "mod" in a control table `package.loaded'.
(So, pre-loaded modules must register themselves in the package.loaded table.)

Otherwise, `require' checks for a function in package.preload["mod"];
if found, it calls that function to load the module.
(So, pre-linked modules must register their open functions in
the package.preload table.)

Otherwise, require looks for a file "mod" in the path package.cpath
(typically it will be something like "./?.so;/usr/local/lib/lua/5.0/?.so"
or "./?.dll;C:windows/?.dll").
If found, it tries to call function "luaopen_mod" on that
library to open it.
Otherwise, it looks for "mod" in the path package.path (old LUA_PATH),
loads it with "loadfile" and runs it.

* It is expected that a module "mod" will put all its stuff in a
global table "mod". Moreover, it will arrange for the
call <<require"mod">> to return that table, so that users can
write

   local xl = require"mod"

and use local xl to access the module.


* Module names can be hierarquical (e.g., "socket.smtp").
A collection of modules with a common root forms a package.
Such hierarchy affects only module names.
The fact that two modules A and B are in the same package
does not create any special relation between A and B.
A module named "mod.lili" should be in a file "mod/lili" in
the path.  Such module should be loaded in global mod.lili.
(Like most strings in this proposal,
the "/" separator can be changed in the luaconfig file.
A system may choose to use a "_" as separator and therefore keep all
modules in a flat directory.)


* The `module' function provides an easy way to create Lua modules
that conform to what is expected from a module.
To write a module, the programmer simply writes

  module(...)     -- ... is the vararg expression

at the start of the chunk and then declares
all module stuff as global values.
The `module' function creates a global with the appropriate name
if it does not exist and sets it as the metatable of
the new module.
It also sets that global as the value of package.loaded[given-name],
so that the corresponding `require' call will return the module.
It also arranges for the module to inherit from _G,
so that it can access (but not change) global values directly.

The `module' function also defines two names in the new namespace:
_NAME has the module name.
_PACK has the package name, which a module can use to require its
own relatives (e.g., <<require _PACK.."mysiblingname">>).


* The `luaL_openlib' function will be changed to put the table
wherein it opens the library in package.loaded[libname].
Therefore, whatever the way the library was open,
a later call to `require' will detect that it is already
open and will return it.
(The standard libraries are a particular example: you can
write <<local s = require"string">> to use the string library.)


* Because everything is written on top of regular Lua stuff (tables,
environments, metatables, etc.) it is easy to bend any rule when there
is a good reason. For instance, if a module exports a single function,
it can return it directly, instead of putting it into a table; a file
may export its stuff into someone else's namespace; more complicated C
modules may be imported through an auxiliary Lua file that calls
loadlib; etc.



--------------------------------------------------------------

package = {}
package.path = LUA_PATH or os.getenv("LUA_PATH") or
             ("./?.lua;" ..
              "/usr/local/share/lua/5.0/?.lua;" ..
              "/usr/local/share/lua/5.0/?/init.lua" )
 
package.cpath = os.getenv("LUA_CPATH") or
             "./?.so;" ..
             "/usr/local/lib/lua/5.0/?.so;" ..
             "/usr/local/lib/lua/5.0/lib?.so"

package.loaded = {}

package.preload = {}


--
-- looks for a file `name' in given path
--
local function search (path, name)
  for c in string.gfind(path, "[^;]+") do
    c = string.gsub(c, "%?", name)
    local f = io.open(c)
    if f then   -- file exist?
      f:close()
      return c
    end
  end
  return nil    -- file not found
end


--
-- new require
--
function _G.require (name)
  if not package.loaded[name] then
    package.loaded[name] = true
    local f = package.preload[name]
    if not f then
      local filename = string.gsub(name, "%.", "/")
      local fullname = search(package.cpath, filename)
      if fullname then
        local openfunc = "luaopen_" .. string.gsub(name, "%.", "")
        f = assert(loadlib(fullname, openfunc))
      else
        fullname = search(package.path, filename)
        if not fullname then
          error("cannot find "..name.." in path "..package.path, 2)
        end
        f = assert(loadfile(fullname))
      end
    end
    local res = f(name)
    if res then package.loaded[name] = res end
  end
  return package.loaded[name]
end


--
-- auxiliar function to read "nested globals"
--
local function getfield (t, f)
  for w in string.gfind(f, "[%w_]+") do
    if not t then return nil end
    t = t[w]
  end
  return t
end


--
-- auxiliar function to write "nested globals"
--
local function setfield (t, f, v)
  for w in string.gfind(f, "([%w_]+)%.") do
    t[w] = t[w] or {}   -- create table if absent
    t = t[w]            -- get the table
  end
  local w = string.gsub(f, "[%w_]+%.", "")   -- get last field name
  t[w] = v            -- do the assignment
end


--
-- new module function
--
function _G.module (name)
  local ns = getfield(_G, name)         -- search for namespace
  if not ns then
    ns = {}                             -- create new namespace
    setmetatable(ns, {__index = _G})
    setfield(_G, name, ns)
    ns._NAME = name
    ns._PACK = string.gsub(name, "[^.]*$", "")
  end
  package.loaded[name] = ns
  setfenv(2, ns)
end


--------------------------------------------------

-- Roberto