Re: Lua registry, environment, and threads.

[Mark Hamburg <mark@...>]

On Jan 7, 2010, at 4:37 PM, Edgar Toernig wrote:

> Roberto Ierusalimschy wrote:
>> 
>>  in t do command end 
>> 
>> is translated to
>> 
>>  do
>>    local _temp = t;
>>    <command>
>>  end
>> 
>> and all acesses to <global> inside command are changed to _temp.global.
> 
> Hm... just my 2c: I was never comfortable with the whole fenv concept
> (each function carries its own table of globals).  IMHO it's too clever
> and produces more confusion then it gives features.  I saw not one use of
> it that did not trigger my "dirty hack" radar.
> 
> I was glad to hear that setfenv&co should be removed in 5.2 and replaced by
> a new "in" statement.  Initially I thought, interesting, dynamic binding for
> globals (like: push a new global table on a stack of globals, pop on "end"
> and "error") but the more I read about the planned changes, the more I think,
> the replacement is not better then the fenv stuff.  It seems to really just
> replace one use case of the setfenv function with syntax (and needs new
> functions for the other use cases).  As the setfenv function is probably
> under the least used functions in the stdlibs, I see no reason which justifies
> that it gets its own syntax.  IMHO, even pcall would make more sense ...
> 
> ET, who only needs two kind of variables: locals and globals, not something
> inbetween.

I've been trying to pull together a more detailed assessment of my reaction to lexical environments, but I'm still reacting and assessing.

I will say that I can appreciate the desire to deprecate setfenv and getfenv. There is a highly-legitimate use for setfenv with respect to sandboxing, but this can better be dealt with as part of the load process. The other uses for these functions seem to be more like the sort of tricks one sees in FORTH leveraging fairly detailed knowledge about the runtime model and as such they feel out of harmony with the rest of the language. One of the things I dislike about the module function changing the environment is that it doesn't have a scope limit. That's fixed with 5.2's lexical environments except that it isn't for compatibility reasons.

Maybe we will start writing object methods as in self do ... end. However, I rather like the explicitness of referencing self because it makes it clear which values are coming from where. Furthermore, since in self blows away all other access to globals, one had better have made sure to import them into lexically scoped locals first. That's arguably a good thing to do in many cases for performance reasons, but it is also a subtlety likely to cause people to spend more time debugging trying to figure out why the call to print in their object method doesn't work. Finally, the code "in self do method() end" will look the method up in self, but it won't pass self to it when it calls it. If one wants to go this route, I think one gets driven fairly rapidly to something like JavaScript's implicit "this" and all of the bizarre behavior that engenders. Just don't go there.

So, that leaves the case of filling out tables with additional values. In addition to the need for global-by-default in a REPL context, config files are probably one of the other big arguments for global-by-default and custom environments for loaded scripts. But is code in the midst of a script really like a config file?

What we seem to be aiming for has more to do with easy table construction. When we build a module, we want to populate the corresponding table with the functions for the module. But if this is really the case, then there are lots of other examples (e.g., Aranha) of how to extend Lua's syntax to make table construction easier. The table scope proposals (http://lua-users.org/wiki/TableScope) also attack some of these problems. Both of these seem more useful than the lexical environments in 5.2 because they preserve the existing environment to at least some extent. At the very least, if this about table construction, then the lexical environments construct should probably be an expression rather than a statement and should return the environment -- e.g., let me write something like:

	package.register "stuff" {

		function foo() end

		function baz() end

		if its_dark_outside() then -- its_dark_outside should be able to be a global reference
			function firefly() end
		end

	}

Actually, if we look at table scopes and we look at extended table construction syntax, what we see is that there is an interesting complementarity between temporarily adding access to an additional set of global names (reading -- e.g., the table scopes case) and making it more convenient to set fields in a table being constructed (writing -- e.g., the module case).

Another direction to look on this front is toward dynamically scoped variables. The reason to read from globals is frequently because it is too cumbersome to pass everything as a parameter. The logical end point for that seems to be something like the PostScript dictionary stack. When we read a global variable, we start at the top of the stack and start looking for a definition. When we write a global variable, we write it into the topmost entry on the stack. This actually turns out to be quite powerful and makes various pieces of code much easier to repurpose because one can redefine the values of the globals they use on a limited basis. I can imagine a really interesting synergy between lexically scoped variables and dynamic variables handled via a table stack. I also, however, don't know how this would interact with sandboxing unless the equivalent of setfenv was assigning the table stack for a function.

As things stand in 5.2, however, the in statement feels like a less successful (because of the problems with completely hiding the containing environment) version of the Pascal WITH statement which Wirth dropped by the time he got to Oberon. (The Oberon WITH does a typecast.)

But maybe I'm just missing the use case and I look forward to being enlightened.

Mark