Re: Determining if _ENV is the first upvalue of a closure?

[Luiz Henrique de Figueiredo <lhf@...>]

The compiler adds _ENV as the first upvalue for the main function of
a chunk. It never worries about that for other functions, and lets
lexical scoping do its work. The only rule the compiler uses is that it
transforms references to global variables v into references to _ENV.v.
Such references are not special in the bytecode. For instance, in the
code below f and g have exactly the same bytecode, except that f has an
upvalue named _ENV. If you strip the debug information, you can't tell
the two functions apart at all.

   local t
   local function f() print(t[1]) end
   local G
   local function g() G.print(t[1]) end

The GETTABUP instruction, which indexes upvalues, is used for both
global variables and fields of other upvalues, so for instance in

   local math=math
   local function f() print(math.sin(2)) end

both print and sin are resolved using GETTABUP, which is a good thing
for size and uniformity of the VM. I think that's the beauty of the
_ENV scheme for resolving global names.

Whether a function has _ENV as an upvalue depends on when and if it ever
refers to a global variable. If it never does, it does not have _ENV as
an upvalue. Even if it does, _ENV may not be the first upvalue.

For instance, consider this code:

   local x
   local function f() return x end
   local function g() print(x) end
   local function h() local y=2*x print(x) end

Then
- f has an upvalue x but not _ENV;
- g has both x and _ENV as upvalues, and _ENV is the first one;
- h has both x and _ENV as upvalues, but x is the first one.

But I'm sure you know all this and I'm not answering your question.

I think the only real solution is to avoid stripping debug information
and search the upvalue list for an upvalue named _ENV.

If you can change the app, then perhaps you can add a special entry
in environments (both the global one and custom ones) and look in
the list of upvalues to see if they are a table with that entry.
This would then work even after stripping debug information.