lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

I am sorry I missed this interesting discussion, and I apologise for joining it a bit late. (I just moved and my ISP was lamentably slow in reconnecting me to the internet.)

1. I personally feel quite strongly that library loading should not automatically insert anything in the global namespace, whether the library in question is a Lua library or a C library. Furthermore, the end user should be protected from knowing details like "what is the entry point of the library in a {.dylib, .so, .dll} file?" So I would like a simple uniform interface like this:

local string = require "string"
coro = require "coroutine"

It does not then bother me that the name I use in the module is "standard"; the require statement includes the standard name.

2. I also believe that it is necessary to distinguish between two use cases:

  1) Module A requires Module B in order to install itself.
  2) Module A will require Module B during execution of its functions.

This allows for modules to have circular dependencies, which seems to be useful. There was a long discussion about this some time ago; although I am not a big fan of circular dependencies, I can see that there are circumstances where they are unavoidable.

3. Finally, I believe it important that it be easy to sandbox module tables. As I have mentioned before, this requires that each sandbox have its own table. It is entirely legitimate to override a module function (or other member), or to add new ones; but sandboxes need to be protected from each other doing this.

One of the wonderful things about Lua is that all of this machinery is quite simple to implement. However, as Diego points out, a little standardisation would help a lot in implementing it.

At a bit more length:

1. Module tables vs. module instantiators

Internally, modules are always generated by executing a function, either a Lua chunk or some lua_CFunction, typically luaopen_*(L). I'm calling this the module instantiator. Currently, the calling conventions for instantiators vary considerably between proposals; my suggestion is that the standard calling convention be that the instantiator is passed a table as its first parameter, which it fills in and returns. The module system then caches both instantiators and tables. The basic logic of require is "if the table has been cached, return it; if the module has an instantiator, use it to create the table; otherwise, find the Lua or C library and get the instantiator from that. A rough implementation is found below.
A simple example in Lua looks like this:

--module complex

local math = _MODULE.math -- see below

return function(complex, name)
  function, imag) return {r = real, i = imag} end
  function complex.add(a, b) return {r = a.r + b.r, i = a.i + b.i} end
  function complex.abs(a) return math.sqrt(a.r^2 +a.i^2) end
  -- etc.
  return complex

In C, this looks about the same (considerably simplified)

/* ... some code at end of message ... */

LUALIB_API int luaopen_complex(lua_State *L) {
  luaL_newmetatable(L, "Complex");
  /* Written out here, but the simplification to luaL_openlib() is obvious */
  lua_pushliteral(L, "new"); lua_pushcfunction(L, complex_new); lua_settable(L, 1);
  lua_pushliteral(L, "add"); lua_pushcfunction(L, complex_add); lua_settable(L, 1);
  lua_pushliteral(L, "abs"); lua_pushcfunction(L, complex_abs); lua_settable(L, 1);
  /* ... */
  lua_settop(L, 1);
  return 1;

The second argument ("name") is provided for autoloaders. This is a slight change from the current autoloader behaviour, where a global is passed to the chunk with the name. Functionally, there is no difference, though.

Note that in the first case, executing the Lua chunk returns an instantiator. In the second case, the instantiator is luaopen_complex. So in both cases, we have an instantiator function to play with. Static libraries (and dynamically generated modules, for that matter) can be registered by adding their instantiator function to the instantiator cache.

The rough implementation below assumes that C module entry points will always be in the form luaopen_<name>, whether or not they are static; this seems like a simple convention and avoids issues with hypothetical OSs which don't like multiply defined dynamically loaded symbols.

(Edgar is quite right that basic version/sanity checks should be performed prior to calling the entry function.)

The advantage of instantiators is that they can be re-executed in different sandboxes, in order to fill independent module tables.
2. Global namespace

I am not so much concerned here about name collision; as Diego (I think) pointed out, modules need to have unique names one way or another. The problem is that there is no reliable way for require() to know which environment table to insert the module into. Furthermore, internal references within the module may or may not be in the same environment as the caller of require(), so hacks like using getfenv to walk up the stack are going to lead to unpredictable results. This is particularly an issue for foreign ("C") modules, which have a different environment than any sandbox. Returning the module table as the result of the require() is much simpler all round.

3. Preloading / circular dependencies

The code below creates _MODULE with a __index metamethod which creates the module table, ready to be filled in. This can be used directly for the case where the library only needs to exist, and will not be called until later. The module tables are then created with a __index metamethod which acts as a trigger; this is based on the implementation in LTN 11.

4. Multiple Lua modules in a single C library.

Some of the base library .c files register more than one Lua module; a couple of them also register globals. In order to get the code presented below to work with this, it will be necessary to at least create separate luaopen_* functions for the different Lua modules. I don't think that is a huge change; the remaining changes to luaL_openlib() actually simplify it.

[CODE SAMPLE 1 -- basic module loading system]
It was easier to write this in Lua, but it needs to be part of the base library in order to get proper integration of static and dynamic libraries. Rewriting it in C would be tedious and arguably unnecessary; it is not going to be of much use in a Lua compiled without at least the byte-code loader, so it can be compiled into the lua executable as bytecode.

-- make a weak keyed table
local function Weak() return setmetatable({}, {__mode = "k"}) end

-- maps module tables to instantiator functions
local MODULE_MAKER = Weak
-- maps module tables to their names
local MODULE_NAME = Weak

-- Find the module. The precise definition of find_in_path might vary,
-- depending on OS interfaces, but a simple implementation would be
-- to try all the possibilities one at a time.

local function find_module(name)
  local filename = find_in_path(name, LUA_PATH)
  if filename then
    return assert(assert(loadfile(filename))())
  filename = find_in_path(name, LUA_DYNPATH)
  if filename then return assert(loadlib(filename, "luaopen_" .. name)) end
  error("Could not find module '""'")

-- An unloaded module lazily instantiates itself; they are created with this
-- metatable.
-- require() forces the instantiation by referencing a module member.
-- I left out the code to verify that the module does not require itself.
local mod_meta = {}
function mod_meta:__index(key)
  local instantiator, name = MODULE_MAKER[self], MODULE_NAME[self]
  if instantiator == nil then
    -- We don't have an instantiator
    instantiator = assert(find_module(name))
    MODULE_MAKER[self] = instantiator
  -- disable the trigger. This should actually change the trigger so that
  -- an error is signalled if the module requires itself, and then set it
  -- to nil after the instantiation
  setmetatable(self, nil)
  -- should do more error checking
  instantiator(self, name)
  return self[key]  -- the trigger has been turned off so rawget is not necessary.

-- The _MODULE table uses the following metamethod to automatically create a new table
-- for so-far-unreferenced modules. The unloaded modules are given the metatable above.

local use_meta = {}
function use_meta:__index(name)
  local mod = setmetatable({}, mod_meta)
  self[name], MODULE_NAME[mod] = mod, name
  return mod

_MODULE = setmetatable({}, use_meta)

function require(name)
  local mod = _MODULE[name]  -- get the module table
  local _ = mod._VERSION  -- trigger the load if necessary
  -- it doesn't matter if mod._VERSION doesn't exist. But it should :)
  return mod

-- Finally, we need a way to register static modules (whether foreign or Lua):

function register_instantiator(name, instantiator)
  local mod = _MODULE[name]
  MODULE_MAKER[mod] = instantiator

-- A C stub for registering static libraries on startup:

LUALIB_API void luaL_registerlib(lua_State *L, const char *name, lua_CFunction instantiator) {
  lua_getglobal(L, "register_instantiator");  /* should check that this worked */
  lua_pushstring(L, name);
  lua_pushcfunction(L, instantiator);
  lua_pcall(L, 2, 0);  /* should do something on error here */

luaL_registerlib(L, "string", luaopen_string);
luaL_registerlib(L, "table", luaopen_table);
/* ... */

[CODE SAMPLE 2 ---- partial C code for Complex example]
#include <math.h>
#include <lua.h>
#include <lauxlib.h>

typedef struct Complex {lua_Number r, i;} Complex;

static int pushComplex(lua_Number r, lua_Number i) {
  Complex *rv = lua_newuserdata(L, sizeof(Complex));
  luaL_getmetatable(L, "Complex");
  lua_setmetatable(L, -1);
  rv->r = r;
  rv->i = i;
  return 1;

static Complex *checkComplex(lua_State *L, int narg) {
  Complex *rv = luaL_checkudata(L, narg, "Complex");
  if (rv == NULL) luaL_typerror(L, narg, "Complex");
  return rv;

static int complex_new(lua_State *L) {
  lua_Number r = luaL_checknumber(L, 1);
  lua_Number i = luaL_checknumber(L, 2);
  return pushComplex(r, i);

static int complex_add(lua_State *L) {
  Complex *a = checkComplex(L, 1);
  Complex *b = checkComplex(L, 2);
  return pushComplex(a->r + b->r, a->i + b->i);

static int complex_abs(lua_State *L) {
  Complex *a = checkComplex(L, 1);
  lua_pushnumber(L, sqrt(a->r * a->r + a->i * a->i));
  return 1;