Re: local "module" pattern without module [was Re: [ANN] SLAXML - pure Lua, robust-ish, SAX-like streaming XML processor]

[Andrew Starks <andrew.starks@...>]

I'm sorry for the double-post-spam, but I read over my last email, and
it was pretty much garbage. I struggled with this issue so much that
I'm going to repost with more verbosity and clarity. Also, I can't
wait to hear other solutions, because I (despite heroic googling)
could not find adequate answers to my issues, other than figuring it
out on my lonesome...

The premise is that you want to make a many-file module, with some of
the files in submodules or otherwise hitherandyon. You do not want to
pollute the global environment and you want to steer clear of
deprecated facilities, such as module.

The first thing to do is to get rid of the global environment
altogether. Remembering that it's only available as long as you don't
reset _ENV, make that your statement! However, constantly re-typing
all of the things that you need from lua into each file is also a
giant, error-prone pain. So, make one file in your module's base
directory the home for your common environment. I call it "_ENV.lua".

Note: You cannot use "init.lua" for this madness, because you need to
be able to load it from your submodules, which are being loaded by
init.lua, which loads your submodules, which are...

My _ENV.lua file, at the risk of being unhelpfully wordy, is the following:

````lua
--_ENV.lua
_ENV = {
	type = type,  pairs = pairs,  ipairs = ipairs,  next = next,  print =
print,  require = require,
	io = io,  table = table,  string = string,  setmetatable =
setmetatable,  assert = assert,
	rawget = rawget, rawset = rawset, error = error,  getmetatable = getmetatable,
	package = package, load = load, loadfile=loadfile, coroutine = coroutine,
	setmetatable = setmetatable, tostring = tostring, tonumber = tonumber;
	pcall = pcall, xpcall = xpcall, select = select,
	lxp = require"lxp", lfs = require"lfs",
	socket = require("socket"), lpeg = require'lpeg',
}

_ENV = require'pl.import_into'(_ENV)
_ENV = tablex.merge(_ENV,{ P = lpeg.P, S = lpeg.S, R = lpeg.R, C =
lpeg.C, Cs = lpeg.Cs,
	Ct = lpeg.Ct, Cf = lpeg.Cf,  Cg = lpeg.Cg,  Cmt = lpeg.Cmt, V = lpeg.V,
	Cc = lpeg.Cc,
	ts = pretty.write, pt = function(t) print(ts(t)) return t end
}, true)

string.strip = stringx.strip
lpeg.locale(lpeg)
eol = P'\r\n' + P'\r' + P'\n'; space = P' '; white_space = space + eol + P'\t'
printf = function(s, ...)  print(string.format(s, ...)) end

return _ENV
````

--[[A shout out to Steve's Penlight package, which makes life
wonderful and to Roberto's LPeg, which has turned regex-hell into a
readable and productive endeavor. I even wrote a validating XML parser
in it, but that's completely unrelated to this thread. :)]]

Anyway, with this file, I now have a common base from which to work.
All of my other modules load this first, using the following command:

     _ENV = require'trms._ENV' --where trms is the base of my module.

This facility was critical for me, for two reasons. First, it kept me
out of global space. If I saw that I was missing something from _G
(took me a surprisingly long time before I saw that I didn't have
tostring!), I would just go back into my _ENV.lua file and add it. As
a required file, this only gets loaded one time, so having it applied
to all of my submodules is 0 calories.

Second, I found it gave me everything that I really needed for using
the "return module as table" protocol, with only a few exceptions. The
biggest exception was when I needed tighter interdependancy between
submodules. This issue has nothing to do with `module` or _ENV... It's
common to all approaches that:

    function my_module:my_method()

....doesn't work when `my_module` was defined in another file. For
example, LuaExpat needs callbacks. These callbacks need access to some
kind of document table. You can put it into the callback, but I didn't
want to do that. Putting all of the callbacks into my my `_init`
method violated my fragile sense of decency. So I passed self into my
`callback.lua` file, which starts off like this:

````callbacks.lua
_ENV = require'trms._ENV'
return function(self)
	local callbacks = {}
	callbacks.StartElement =  function(parser, elementName, attributes)
		local res = {}
		local stack = self.stack
        ---awesome stuff for about 150 lines...
        return callbacks
end
````
To use this, I simply call the return value of require when assigning
the callback table to my parent module, like so:

    --back in init.lua
    self.callbacks = require'trms.xml.callbacks'(self)

Now, I get my callbacks table out of my initializer and `callbacks`
has access to the state, as it needs to.

Most often, I don't need to do this. If I'm passing state or self
between submodules, I'm almost always doing it wrong. My internal
policy is that if I'm doing something that is highly-related to
another file, I *might* be okay. More likely, I'm putting something in
the wrong spot and there is a way to do it without passing anything
between modules.

The one limitation that I don't have an answer for is loading a parent
module from its child. I mean, I could, if I wanted to spend way too
much time figuring it out, but I gave up and now I don't do it. I just
test from the root module responsible for loading the others.

This all works really well for me, it's organized and I don't think
about it anymore. Hopefully I'm not missing some huge problem and if I
am, I'm ready to accept the bad news (and then ignore it until my
world falls apart). I'd also love to hear other solutions to module
management in the 5.2 era.

-Andrew Starks