Re: a new proposal for environments

[David Manura <dm.lua@...>]

On Mon, Mar 1, 2010 at 3:01 AM, Juris Kalnins wrote:
> It's much easier to reason about code when the whole concept of globals and
> environments is gone from the internals.

Come to think of it, function environments were one of the few things
my lua2c and source optimizer [1-2] didn't handle very well.

For example, optimizing `local function f() return x end; return f()`
to `return x` might not in general be a valid transformation.  This
could occur if `f` has an environment, which may be difficult to infer
if the environment is attached implicitly at the time of `f`'s
construction rather than via an explicit setfenv on `f`.  If may also
occur if accessing the global `x` triggers a metamethod that does a
getfenv or setfenv with a stack level parameter because the stack
levels have been adjusted in the transformation.  That problem is more
general than just environments though and affects any function that
that may read stack levels.  Fortunately, with the removal of
getfenv/setfenv, these functions are now more rare, mainly in `error`,
where at least stack level is normally just advisory, and in the debug
library.  I've previously mentioned the difficulties that stack level
parameters impose [3].  Correctness of the above inlining may require
eliminating stack level parameters in non-debug code.  Another
solution may be to patch Lua to accept a new construct that increases
or decreases the stack level within a scope (`stacklevel 1 do return x
end`), but the code might still not be exactly equivalent since, for
example, the debug API may see that some stack levels now aren't
attached to objects of type 'function'.

The problems that lua2c had with environments were mainly in that
getfenv/setfenv didn't accept C functions.  This is a problem because
lua2c translates Lua functions into C functions: `local function f()
end` translates to roughly `int lua2c_f(lua_State * L) { ..... }`.
Maybe this is not a critical problem since you could override these
functions to accept C functions.  Alternately, lua2c might wrap all
generated C functions in Lua functions, although then we again have
the problems with adjustments to stack levels.  Anyway, it's good that
these workarounds can now be avoided.  I'm not sure if there are other
functions in Lua that accept Lua functions but reject C functions.

Incidentally, lua2c can handle Lua code with upvalues, which includes
the new _ENV thing, although its upvalue handling is inefficient.  The
problem is how to take Lua code containing two functions that share an
upvalue and translate that into two C functions that share a Lua
variable.  There were some discussions of allowing C functions to
share upvalues [4].   lua2c took the workaround suggested of
implementing shared upvalues in C by storing them in a table that in
turn is placed on the stack of each C functions.  This imposes the
cost of table lookup for "upvalue" access and, more significantly, the
cost of constructing temporary tables for upvalue storage on the
stack.  A single global table for upvalue storage would not do since
it would not be properly cleaned up upon an exception.  This is
probably the main thing that holds back the performance of lua2c.

[1] http://lua-users.org/wiki/LuaToCee
[2] http://lua-users.org/wiki/SourceOptimizer
[3] http://lua-users.org/lists/lua-l/2010-01/msg01255.html
[4] http://lua-users.org/lists/lua-l/2006-08/msg00189.html