The compiler adds _ENV as the first upvalue for the main function of
a chunk. It never worries about that for other functions, and lets
lexical scoping do its work. The only rule the compiler uses is that it
transforms references to global variables v into references to _ENV.v.
Such references are not special in the bytecode. For instance, in the
code below f and g have exactly the same bytecode, except that f has an
upvalue named _ENV. If you strip the debug information, you can't tell
the two functions apart at all.
local t
local function f() print(t[1]) end
local G
local function g() G.print(t[1]) end
The GETTABUP instruction, which indexes upvalues, is used for both
global variables and fields of other upvalues, so for instance in
local math=math
local function f() print(math.sin(2)) end
both print and sin are resolved using GETTABUP, which is a good thing
for size and uniformity of the VM. I think that's the beauty of the
_ENV scheme for resolving global names.
Whether a function has _ENV as an upvalue depends on when and if it ever
refers to a global variable. If it never does, it does not have _ENV as
an upvalue. Even if it does, _ENV may not be the first upvalue.
For instance, consider this code:
local x
local function f() return x end
local function g() print(x) end
local function h() local y=2*x print(x) end
Then
- f has an upvalue x but not _ENV;
- g has both x and _ENV as upvalues, and _ENV is the first one;
- h has both x and _ENV as upvalues, but x is the first one.
But I'm sure you know all this and I'm not answering your question.
I think the only real solution is to avoid stripping debug information
and search the upvalue list for an upvalue named _ENV.
If you can change the app, then perhaps you can add a special entry
in environments (both the global one and custom ones) and look in
the list of upvalues to see if they are a table with that entry.
This would then work even after stripping debug information.
