Re: Determining if _ENV is the first upvalue of a closure?

[Philipp Janda <siffiejoe@...>]

Am 13.03.2015 um 05:28 schröbte Tim Hill:
> For various reasons we need to dynamically move Lua functions
> (actually closures) across lua_State boundaries. This is of course
> easy using dump()/load() pairs, portability of the bytecode is not a
> problem since this is all occurring in-process on a given
> architecture.
>
> Of course, dump()/load() doesnt handle any upvalues for the closure
> (ok, function) being dumped, no surprises there. However, since we
> DO need to transport some upvalues with certain functions, we
> serialize them and send them along with the dumped function. This all
> works fine, after loading the closure at the receiver we simply
> de-searialize the upvalues are install them in the loaded closure.
>
> The problem occurs with _ENV. Of course we don’t serialize the
> environment (nor could we), so we skip the first upvalue. However,
> not all functions *have* _ENV as the first upvalue, since the
> compiler only emits this if it is used in the function.
>
> The problem then, is determining which functions have _ENV as an
> upvalue and which do not. I’m unable to find a deterministic way to
> do this. I can’t look at the type of the first upvalue, since some of
> our upvalues are also tables. I can’t look at the name of the first
> upvalue, since this can be stripped out with debug information.

Some third-party modules (e.g. penlight.compat or busted) rely on 
upvalue names to be present anyway, so maybe that is a reasonable 
assumption/restriction.

> I *can* look to see if the first upvalue is the same table as the
> distinguished environment, but (argh) some functions have custom
> environments and hence a different environment.

But in that case you probably need to serialize that custom environment 
anyway, so where's the problem?! Actually, not serializing the 
distinguished global environment is just a special case of a Lua value 
registry for predefined values: You could scan `package.loaded` (maybe 
even dynamically by monkey-patching `require`) and send the module name 
and the "path" to the value instead of the serialized value itself. On 
the receiver side you `require` the module name and retrieve the value 
by following the "path". This works for things like `io.stdout`, or 
module tables, or functions in module tables.

> I can look to see if the table has _G (and if it is a self-reference
> to the table), but while this is a good hint it is of course not
> deterministic as a regular table could have this (though it would be
> odd).
>
> Clearly the compiler knows is it emitting a closure (ok, function
> prototype) with _ENV as the first upvalue, since it makes sure _ENV
> is always the first upvalue. But it doesn’t seem to leave any
> breadcrumbs that I can access from the C API to determine this for a
> given function.
>
> Any ideas anyone???
>
> —Tim

Philipp