Using lexical environments (was setfenv/getfenv)

[Nevin Flanagan <Alestane@...>]

On Jan 12, 2010, at 1:13 PM, Petite Abeille wrote:

> [edited for brevity]
> 
> On Jan 12, 2010, at 4:25 PM, Roberto Ierusalimschy wrote:
> 
>> The main motivation was that, in our view, setfenv/getfenv was too
>> dynamic.
> 
> Hmmm... since when is there such a thing as "too dynamic" in Lua?
> 
>> In other words, almost anything can change the environment of anything.
> 
> Yes, and?
> 
>> We tried to restrict such uses.
> 
> Why? We are all consenting adults, no? 

I had the dubious pleasure last year to do a great deal of work designing a sandbox for Lua code. The system was capable of loading user-supplied code both for execution inside and execution outside the sandbox; code inside the sandbox had higher privileges but restricted access to information.

The usefulness of get/setfenv outside the sandbox made it almost impossible to suitably control the flow of restricted information into the sandbox. In the end we had to modify luaB_getfenv and luaB_setfenv to respect a metafield "__environment" on the tables set as environments.

I strongly favor the transition to lexical environments. What I think would be the most versatile addition to this would be the addition to the lua_ API  of a function such as "lua_pushclosurecopy" or something similar, that would create on the stack a copy of the closure at a given index, with the same code and upvalues but as a separate object that could be passed or collected separately. This would make it trivial to add a function inside Lua that would create a copy of a closure with its own environment, without decompiling or recompiling it. This would also mean that code could keep a closure without the fear that a new environment could be forced on it, since it would not know about the copy.

However, there is another solution as well:

local funcNeedsDynamicEnv, mutator
do 
	local environment;
	funcNeedsDynamicEnv = function (someArguments)
		in environment do
			 return operationOnArgumentsUpvaluesAndGlobals();
		end
	end
	mutator = function(env) environment = env end
end

(A number of other possible constructs can have similar effects.) 

In other words, if you want a function to be executable with a dynamic environment, build it that way. However, I still endorse the "clone a closure" solution for cases where you might be creating an infrastructure that runs extension code in dynamically changing environments, and cannot guarantee that their code will expect an environment argument or contain an "in" block.


My biggest concern remains the challenge of getting values in or out of an "in" block. Right now the only ways to get a value out are to either extract it from the environment table or create a local before the block where it can leave its result; is there any hope for a way to pass values in and out via the stack, much as coroutine.resume and coroutine.yield do?

This would also help us solve the problem of wrapping code that is not informed about a specialized environment but expects arguments:

exprList = in (...) with select(2, ...) do
  body;
with resultExprs end

This would cause "..." as used within the "in" block to evaluate to the first "..." list, minus the first argument. That is, the expression list between "with" and "do" would be accessible within the block as "...". The "in" block would effectively become its own expression that evaluated to the expressionList resultExprs.

Both "with" clauses would be optional.

I considered using "return" but there are ambiguity issues. I think this should be unambiguous to the parser, usable to the coder, and teachable to the student, which makes it a hat trick.

I am of course pleased to hear discussion.

NFF