lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

Hey Lua team!

I had this idea some years ago, I call it the "environment stack". It's perfectly compatible with existing Lua syntax, requiring only changes to the code generator, VM and libraries, and eliminates one of the pain points of Lua: metatable handling. (I've been told you plan on changing metatable handling, so I thought I'd pitch in)

Current Lua uses an actual register/upvalue for _ENV, and uses register/upvalue lookup to find _ENV. Thus, code of the form:


First transforms into:


Which then transforms into:

<register or upvalue for _ENV>.print(<register or upvalue for _ENV>.getmetatable(""))

I am not proposing we change the first transformation. Instead, I am proposing we change the second transformation.

What the "environment stack" does is, it provides a stack of environments.

With the 6 opcodes: GETTABST, SETTABST, PUSHST, POPST, GETST, SETST, we can build an "environment stack". From the given Lua code:

  local _ENV = {print=print}

We emit the following opcodes, with added comments detailing code generator behaviour, as well as explaining how the opcodes operate:

NEWTABLE 0 0 1 ; creates table at reg 0
GETTABST 1 0 -1 ; gets key "print" from upvalue 0, puts it at reg 1
SETTABLE 0 -1 1 ; sets key "print" at reg 0 to value at reg 1
; (frees reg 1)
PUSHST 0 0 ; pushes table at reg 0 onto "environment stack" slot 0
; (frees reg 0)
GETTABST 0 0 -1 ; gets key "print" from "environment stack" slot 0, puts it at reg 0
CALL 0 1 1 ; calls function at reg 0
; (frees reg 0)
POPST 0 ; pops "environment stack" slot 0 onto reg 0 -- corresponds to "end" in above example

This can then be easily extended to metatables:

_ENV --> slot 0
_MT_STRING --> slot 1
_MT_NUMBER --> slot 2
and so on

On a simple block of code like the above, this is indeed a simple push/pop mechanism. However, if we add in some closures:

local f
local g
  local _ENV = {}
  function f() return _ENV end
  function g(newenv) _ENV=newenv end
print(f()) -- prints 1

Up until "local _ENV = {}" it's pretty simple: make a table. push it. However, when you create the closure, the CLOSURE opcode captures the current values/slots on the stack(s). This is similar to how upvalues work, really. As with upvalues, you can set the environment (with _ENV = whatever) in the closure and it'll change the environment for everything using the same environment.

A small edge case: you can currently accept _ENV as an argument:

local function f(_ENV)
f({print=function() print "hello world" end})

This is a rare edge case, but easily handled: just emit a PUSHST as the first instruction in a function.
As for repeating it:

local function f(_ENV, _ENV, _ENV) end

Same idea applies, just PUSHST every one (from left to right, please!).

As for metatables and things: every time you use an opcode, it has access to the current environment stack. It can easily look up which metatable it needs to use. On the other hand, getmetatable and setmetatable now strictly only apply to tables and full userdata - all other metatables are accessed through the reserved names _MT_*.

To sum it up, here are some cases and their handling:

function f() local _ENV = {} return _ENV end -- handled by above rules - "return _ENV" generates a GETST for _ENV, then "return" itself pops all relevant PUSHSTs, and then does RETURN.
do local _ENV = {} return _ENV end -- same as above
do local _ENV = {} end -- simple PUSHST followed by POPST
function f(_ENV) end -- has to generate additional PUSHSTs for each _ENV in the argument list. do ::test:: local _ENV = {} goto test end -- "goto test" needs to POPST. perhaps similar to `do ::test:: local f; local function g() return f end; goto test; end`.
-- (most cases can be compared to closures and upvalues)

And here are some benefits:

- Modules can use things from caller environment using debug library. (I've seen modules do this with coroutines before. See below.) - SIGNIFICANTLY simplifies virtualization modules. Previously I've used coroutines for this: I installed global metatables for all types (using debug.setmetatable), then used the currently running coroutine to figure out which environment made the call (with `metatables[coroutine.running()][currentType]`), then passed it on to the correct function. This feature would completely eliminate all that code! - Modules would be able to locally customize string, number, etc metatables, without affecting the main program. Modules will finally be able to use `"%s %s" % {"hello", "world!"}`!
- Among other things.

I'm not providing an implementation because I know you don't accept patches. However, if you do this, please don't make the Lua 5.1 mistake of implementing this as a call-stack-sensitive standard library function - if we've learned anything from Lua 5.1 it's that it's unsafe and dangerous, and makes sandboxing rather difficult.

Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.