Re: Search paths in require/packages

[Jerome Vuarand <jerome.vuarand@...>]

2013/12/6 Robert Virding <robert.virding@erlang-solutions.com>:
> That sounds definitely weird. How do I then tell require *where* to look for the module? So if I write "a = require 'foo'" how does does the loading mechanism know where to look for 'foo'? The default path values imply that requiring a 'foo' will try to find either a foo.lua or an foo/init.lua in a number of different places. This is also described in the package.searchers description, both by referencing package.searchpath() for Lua loader and very explicitly for the C library loader.

Maybe we should recap each term:

A module can be two things depending on the point of view. A module
can be the runtime object that represent a module in the Lua
interpreter. It's usually a table containing functions, but it can be
any Lua value. A module can also be the Lua file that defines the
module (or the DLL file that expose the module). For the sake of
clarity, let's call the first thing the "module object" and the second
thing the "module source" (even though it might not always be source
code). All modules have a "module object", but some may not have a
"module source".

A loader is a function, that when called, will return a module object.
luaopen_string is a C loader. The chunk/function resulting from the
compilation of socket.lua is a Lua loader.

A searcher is a function that, given a module name, will look for a
module source with that name, and create and return a loader for that
module. There are 4 searchers in a standard Lua interpreter, you can
add more or remove any of these 4. For example the Lua searcher will
look for .lua files as module source, and compile them (but not
execute them).

require is a function that calls searchers, calls loaders, and manages modules.

> So if the paths are for finding loaders how do I tell where to find the modules? Does that mean I have to give the absolute path for require?

The paths are to find files, the module source. Then the Lua searcher
compiles the file, and return the result of the compilation, that is
the loader for the module. Then require will execute the file (it's
just a function, require doesn't know it comes from a file), which
returns the module table.

Here is a short implementation (without error handling) of all these
concepts in Lua. Hopefully it will make things a bit clearer.

local function lua_searcher(modname)
    local filepath = package.searchpath(modname, package.path, '.', '/')
    if not filepath then
        return nil
    end
    local loader = loadfile(filepath)
    return loader
end

local searchers = {
    preload_searcher,
    lua_searcher,
    c_searcher,
    all_in_one_searcher,
}

function require(modname)
    -- check if the module has already been loaded
    if not package.loaded[modname] then
        local loader
        -- call each searcher to find a loader
        for _,searcher in ipairs(searchers) do
            loader = searcher(modname)
            if loader then break end
        end
        -- execute the loader
        local module = loader(modname)
        -- store the module
        package.loaded[modname] = module
    end
    -- return the module
    return package.loaded[modname]
end

If you answer this please use "module object" or "module source" to
clearly distinguish what you mean.