Lua Hacks

wiki

In modern computer programming, a "hack" can refer to a solution or method which functions correctly but which is "ugly" in its concept, which works outside the accepted structures and norms of the environment, or which is not easily extendable or maintainable (see kludge)....In a similar vein, a "hack" may refer to works outside of computer programming. For example, a math hack means a clever solution to a mathematical problem.

Hacks can be less elegant, less practical, and less recommended than LuaDesignPatterns. However, they are still interesting for academic value. They may lead to more elegant solutions or provide useful inspiration on other problems.

Each hack is identified with a name and contains a description of what it tries to do and how it does it.

Some examples use getfenv or setfenv, which were retired after Lua 5.1.

Hacks on other Pages

• ExpressionsAsStatements - rewriting statements as expressions
• SwitchStatement - lists implementations of C-style switch statements
• BoundScalarGlobalsOne - defining global variables that invoke C functions on get/set
• MakingLuaLikeRuby - simulating some of the semantics of Ruby in Lua
• CustomOperators - defining new operators beyond the built-in ones
• StatementsInExpressions - reusing subexpressions in an expression
• ExpressionTemplatesInLua - a Lua coding technique similar to C++ expression templates
• GettingVariablesFromValues - gets variables of values passed to a function (e.g. pass-by-reference)
• RuntimeSyntax - automates the run-time simulation of what resembles syntax extensions

Hack: Modules a la Perl

The following example makes Lua behave like Perl's Exporter/import thing for exporting module variables into the caller namespace.

-- file test.lua
local mymodule = require("mymodule")
assert(hello == nil) -- not imported
mymodule.hello()

local mymodule = require("mymodule"):import{"hello"}
hello()
assert(goodbye == nil) -- not imported
assert(mymodule.print == nil) -- not visible
 
require("mymodule"):import{":all"}
goodbye()

-- file mymodule.lua
 
-- Globals get collected by this table.
local collector = {}
setfenv(1, setmetatable(collector, {__index = _G}))
 
-- Now define the module functions.
 
function hello()
  print("hello?")
end
function goodbye()
  print("goodbye?")
end
 
-- Used to import functions into caller namespace
-- (like in Perl's "import" function in Exporter.pm)
function import(self, option)
  -- convert any option list to set
  local optionset = {}
  for _,v in ipairs(option) do optionset[v] = true end
  -- import selected variables into caller namespace
  local env = getfenv(2)
  for k,v in pairs(collector) do
    if optionset[":all"] or optionset[k] then env[k] = v end
  end
  return self
end
 
-- Return public module object (don't expose globals)
local mymodule = {}
for k,v in pairs(collector) do mymodule[k] = v end
return mymodule

-- output
hello?
hello?
goodbye?

Note: It's preferrable to use things like mymodule = requre "mymodule"; local hello = mymodule.hello instead, which only import a single symbol into your namespace (cleaner). import{":all"} has similar uses and disadvantages of Java's static import.

--DavidManura, 2006-10, Lua 5.1

Hack: Dynamically Scoped Variables

Warning: the following is academic and isn't really recommended for most situations.

Back in the good-ol'-days, before the popularity of static scoping, we had dynamic scoping ([Wikipedia:Scope_(programming)]). Let's simulate that in Lua.

To do so, we can create a function that wraps the provided function to apply dynamic scoping to certain variables:

-- (func, varnames*) --> func
function dynamic(func, ...)
  local varnames = {...}
  local saves = {}
  return function(...)
    for _,varname in ipairs(varnames) do saves[varname] = _G[varname] end
    local result = pack2(func(...))
    for _,varname in ipairs(varnames) do _G[varname] = saves[varname] end
    return unpack2(result)
  end
end

The above function uses the pack2 and unpack2 functions from the "Vararg Saving" design pattern in VarargTheSecondClassCitizen. dynamic saves and restores the values of the given global variables before and after the function call, thereby simulating dynamic variables. Here's an example usage:

test2 = dynamic(function()
  print("test2:", x, y)
  x = 6
  y = 7
  return x, y
end, 'y')

test1 = dynamic(function()
  print("test1:", x, y)
  x = 4
  y = 5
  print("test2:", test2())
  print("test1:", x, y)
end, 'x')

x = 2
y = 3
test1()
-- Output:
--   print("main:", x, y)
--   test1:  2       3
--   test2:  4       5
--   test2:  6       7
--   test1:  6       5
--   main:   2       5

Note: languages that support RAII ([Wikipedia:Resource_Acquisition_Is_Initialization]) can implement this with RAII. There may be some ways to simulate RAII in Lua (LuaList:2006-09/msg00846.html).

The above code does not properly handle functions that raise exceptions. That would require inserting a pcall into dynamic.

--DavidManura, 2007-01

Pattern: Local by Default Scoping

Warning: the following is academic and isn't really recommended for most situations.

In Lua, if you assign a value to an undefined variable, the variable is created as a global rather than a local. Let's change this to make variables local by default (like in Python [2]). By "local" we mean in the sense of smallest lexical scope, not in the Lua implementation sense of being stored in the Lua stack (here, we internally implement locals with globals on the heap).

As in the previous "Dynamically Scoped Variables" design pattern, we create a utility function that wraps other functions that we want to assign this behavior to and makes use of the pack2 and unpack2 functions:

function localbydefault(func)
  local upenv = getfenv(2)
  local mt = {}; mt.__index = upenv
  return function(...)
    local env = setmetatable({}, mt)  -- storage for locals

    local oldenv = getfenv(func)
    setfenv(func, env)
    local result = pack2(func(...))
    setfenv(func, oldenv)
    return unpack2(result)
  end
end

This causes a temporary environment for locals to be created for each function call.

Example:

test2 = localbydefault(function()
  print("test2:", x, y)
  x = 6; y = 7
  _G.z = 8
  return x, y
end)

test1 = localbydefault(function()
  print("test1:", x, y)
  x = 4; y = 5
  print("test1:", x, y)
  print("test2:", test2())
  print("test1:", x, y)

  localbydefault(function() -- nested
    print("test3:", x, y)
    x = 9; y = 10
    print("test3:", x, y)
  end)()
  print("test1:", x, y)
end)

x = 2
test1()
print("main:", x, y, z)
-- Output:
--   test1:  2       nil
--   test1:  4       5
--   test2:  2       nil
--   test2:  6       7
--   test1:  4       5
--   test3:  4       5
--   test3:  9       10
--   test1:  4       5
--   main:   2       nil     8

Note how globals can be accessed via the _G variable.

This approach works for recursive functions too:

fibonacci = localbydefault(function(n)
  if n == 1 then return 0
  elseif n == 2 then return 1
  else
    x1 = fibonacci(n - 1)
    x2 = fibonacci(n - 2)
    return x1 + x2
  end
end)
assert(fibonacci(10) == 34)

The above function was written to use temporary variables in such a way that it would fail if those temporary variables were globals rather than locals (e.g. try removing the localbydefault from that). However, x1 and x2 are indeed lexically scoped locals, and the function works.

-- David Manura, 2007-01

//Comment: What is pack2 and unpack2

Hack: Mimicking C++ Iostreams

Warning: the following is academic and isn't really recommended for most situations.

C++ iostreams [3] are used as such:

#include <iostream>
using namespace std;
int main()
{
    cout << "hello" << "world" << 123 << endl;
    return 0;
}

We can mimick that in Lua as such.

cout = function (str)
  if str ~= nil then
    io.write(tostring(str), " ")
  else
    io.write("\n")
  end
  return cout
end

Example usage:

cout "hello" "world" (123) () --> "hello world 123\n"

You might even add sticky formatting functions too:

cout "asdf" (intfmt(3)) (i)

See also SimpleStringBuffer for a related example.

Hack: Accessing Lexical Variables in the Caller / Lexical String Interpolation

Warning: the following is academic and isn't really recommended for most situations.

As shown in StringInterpolation, one can define a function that access variables lexically scoped in its caller:

local x = 3
assert(interp "x = ${x}" == "x = 3")

This is done via the debug.getlocal.

Another application for this might be to eliminate the need for passing locals in "Stringified Anonymous Functions" in ShortAnonymousFunctions.

--DavidManura

Hack: Modifying Bytecode

This example modifies bytecode at run-time. Possibly more useful tricks could be based on it. Note that string.dump does not preserve up-values, which limits the usefulness of this (but see also PlutoLibrary).

function add(x,y) return x + y end
function test(x,y)
  print("here is waht the answer is...")
  print("teh answer is", add(x,y))
end
 
local replacements = {
  ["teh"] = "the",
  ["waht"] = "what"
}
function fix_spelling()
  local env = getfenv(2)
  for k,v in pairs(env) do
    if type(v) == "function" then
      local success, bytes = pcall(function() return string.dump(v) end)
      if success then
        local is_changed, n = false, nil
        for k,v in pairs(replacements) do
          bytes, n = bytes:gsub(k, v)
          is_changed = is_changed or (n > 0)          
        end
        if is_changed then
          env[k] = assert(loadstring(bytes))
        end
      end
    end
  end
end
 
fix_spelling()
 
test(2,3)

$ lua test.lua
here is what the answer is...
the answer is   5

--DavidManura, 2007-03

Hack: Proxy Table of Local Variables, _L

Warning: this example is academic and not really intended for production use.

Here's how we might create a proxy table that does reading/writing of local variables an get/set. Internally it uses debug.getlocal and debug.setlocal calls, which is what makes this a hack.

-- Returns a proxy table representing all locals visible to the
-- given stack level <level>.  The table is readable
-- and writable (writing modifies the local).
--
-- If <level> is nil, the default used is 1,
-- which indicates the stack level of the caller.
--
-- NOTE: This function is based on debug.getlocal()
-- and may be slow.
do
  local Nil = {} -- placeholder (for storing nils in tables)
  local function getstackdepth(level)
    local n = 1
    while debug.getinfo(n+level+1, "") ~= nil do n=n+1 end
    return n
  end
function getlocals(level)
  level = (level or 1) + 1

  -- Note: this correctly handles the case where two locals have the
  -- same name: "local x=1; local x=2 ... get_locals() ... local x=3".

  local mt = {}
  local proxy = setmetatable({}, mt)
  local levels = {}   -- map: variable name --> stack level
  local indicies = {} -- map: variable name --> stack index

  local depth = getstackdepth(level)

  for k=1,depth do
    -- Determine number of locals (nlocs)
    -- Note: it would be easier if debug.getinfo returned nlocs.
    local nlocs = 0
    while debug.getlocal(level, nlocs+1) do nlocs = nlocs + 1 end

    -- Record locations of locals by name.
    for n=nlocs,1,-1 do
      local lname, lvalue = debug.getlocal(level, n)
      if lvalue == nil then lvalue = Nil end  -- placeholder
      if not levels[lname] then  -- not shadowed
        levels[lname] = level
        indicies[lname] = n
      end
    end
    level = level + 1
  end

  -- proxy handlers for read/write on table.
  function mt.__index(t, k)
    local depthdiff = getstackdepth(2) - depth
    if depthdiff < 0 then error("stack out of scope", 2) end
    local level = levels[k]
    local v
    if level then
      level = level + depthdiff  -- a correction
      local _; _, v = debug.getlocal(level, indicies[k])
      if v == Nil then v = nil end
    end
    return v
  end
  function mt.__newindex(t, k, v)
    local depthdiff = getstackdepth(2) - depth
    if depthdiff < 0 then error("stack out of scope", 2) end
    local level = levels[k]
    if level then
      level = level + depthdiff  -- a correction
      debug.setlocal(level, indicies[k], v)
    end
  end

  -- note: "stack out of scope" could be made more robust (see test suite)

  return proxy
end end

-- test suite
function test()
  local function update(L)
    assert(L.x == 10)
    L.x = 20
  end

  local L2

  local x = 1
  local y = 3
  local z = 5
  function f1()
    local x = 2
    local y = nil
    local x = 4
    local _L = getlocals()
    assert(_L.w == nil)
    assert(_L.x == 4)
    assert(_L.y == nil)
    assert(_L.z == 5)
    _L.z = 6  -- modify local through table!
    assert(z == 6 and _L.z == 6)
    z = 7
    assert(z == 7 and _L.z == 7)
    _L.x = 10
    assert(x == 10 and _L.x == 10)
    update(_L)
    assert(x == 20 and _L.x == 20)

    L2 = _L

    local x = 5  -- doesn't interfere
  end
  f1()

  -- Note: L2 is invalid at this scope.
  -- print(L2.x)  -- will correctly raise an error
  -- L2.x = 1  -- will correctly raise an error
  -- update(L2) -- opps, this doesn't trigger "stack out of scope"

  print "done"
end
test()

--I did not write the above, I'm a totally different person--

I really find this page interesting. I read through these trying to understand them. I didn't understand the title or the code with this one, so I decided to skip over it. Now just recently I did exactly what this code does, but shorter, and more efficient.

Using this code, _L always exist, it does not have to be redefined over and over with '_L = getlocals()' it is also readable and writeable. It is much simpler to understand and not as overcomplicated as the code above. The code is as follows;

--Setup--
setlocal = function(lvl, name, val)
   local setlocal = saved
   if lvl ~= 0 then
      lvl = lvl + 1
   end
   local i = 1
   while true do
      local var, value = debug.getlocal(lvl, i)
      if not var then
         break
      end
      if var == name then
         debug.setlocal(lvl, i, val)
      end
      i = i + 1
   end
end
--End of setup--

--Code--
setmetatable(_G, {
   __index = function(tab, var, val)
      if var == "_L" then
         local variables = {}
         local idx = 1
         while true do
            local ln, lv = debug.getlocal(2, idx)
            if ln ~= nil then
               variables[ln] = lv
            else
               break
            end
            idx = 1 + idx
         end
         return setmetatable({}, {
            __index = variables,
            __newindex = function(tab, var, val)
               rawset(variables, var, val)
               setlocal(2, var, val)
            end
         })
      end
   end
})
--End of code--

--Tests--
local a = 1
print(a)
_L.a = 5
print(a)
print(_L.a)
--End of tests--

I feel as if I should give credit to u0b34a0f6ae from stackoverflow for providing part of the setlocal function, which he posted here; "http://stackoverflow.com/questions/2834579/print-all-local-variables-accessible-to-the-current-scope-in-lua" (It was marked as best answer)

I'm currently trying to implement a metatable for _L comment underneath here if you have any ideas on how I could do this.

Hack: User-defined Control Structures and Keywords

The following simulates user-defined control structures and keywords, allowing a certain level of metaprogramming. (See also RuntimeSyntax.)

local Expr = {}

local function ev(o) -- terminate and eval expression
  if getmetatable(o) == Expr then return o.END() else return o end
end
function Expr.__unm(a) return -ev(a) end
function Expr.__len(a) return #ev(a) end
function Expr.__add(a, b) return ev(a) + ev(b) end
function Expr.__sub(a, b) return ev(a) - ev(b) end
function Expr.__mul(a, b) return ev(a) * ev(b) end
function Expr.__div(a, b) return ev(a) / ev(b) end
function Expr.__pow(a, b) return ev(a) ^ ev(b) end
function Expr.__concat(a,b) return ev(a) .. ev(b) end
function Expr.__eq(a,b) return ev(a) == ev(b) end
function Expr.__lt(a,b) return ev(a) < ev(b) end
function Expr.__le(a,b) return ev(a) <= ev(b) end
function Expr.__index(a,b) return ev(a)[ev(b)] end
--function Expr.__newindex(a,b,c) ev(a)[ev(b)] = ev(c) end
function Expr.__call(o, ...)
  if ... == ';' then return o.END()
  elseif o.__CALL then return o.__CALL(...)
  else return ev(a)(...) end
end

function Expr:clear()
  for k,v in pairs(self) do self[k] = nil end
end

local function eval(t)
  if type(t) == "function"  or getmetatable(t) == Expr then return t() end

  local s = ""
  local ts = {}
  local vals = {}
  if type(t) == "table" then
    for k,v in pairs(t) do
      if type(k) ~= "number" then
        vals[#ts+1] = v
        ts[#ts+1] = k
      end
    end
    t = t[1]
    s = ((#ts > 0) and "local " .. table.concat(ts, ",") .. " = ...; " or "")
  end
  local s = s .. "return " .. t
  local f = loadstring(s)
  return f(unpack(vals, 1, #ts))
end
--print(eval {[[x+y]], x = 2, y = 5})

function Expr.EVAL (expr)
  return eval(expr)
end

function Expr.IF (cond)
  local o = setmetatable({}, Expr)
  function o.THEN (stat)
    o:clear()
    function o.END ()
      if (eval(cond)) then
        eval(stat)
      end
    end
    o.__CALL = o.END
    return o
  end; return o
end

function Expr.LET (name)
  local o = setmetatable({}, Expr)
  function o.EQUAL (expr)
    o:clear()
    function o.END ()
      _G[name] = eval(expr)
    end
    function o.IN (expr2)
      o:clear(); function o.END()
        local oldval = _G[name]
        _G[name] = eval(expr)
        local function helper(...)
          _G[name] = oldval
          return ...
        end
        return helper(eval(expr2))
      end; return o
    end; return o
  end
  o.__CALL = o.EQUAL
  return o
end

function Expr.FOR (var)
  local o = setmetatable({}, Expr)
  function o.FROM (i1)
    o:clear(); function o.TO (i2)
      o:clear(); function o.DO (expr)
        o:clear(); function o.END()
          for n=eval(i1),eval(i2) do
            _G[var] = n
            eval(expr)
          end
        end; return o
      end; return o
    end; return o
  end
  return o
end

Expr.__index = Expr

setmetatable(_G, Expr)

-- TEST

LET 'x' .EQUAL '1' ';'
LET 'y' .EQUAL '3' ';'
IF 'x == 1' .THEN 'print(x+2)' ';'
FOR 'n' .FROM '1' .TO '3' .DO (
  IF 'n > 1' .THEN 'print(n,x)'
) ';'
print(1 + (LET 'x' '2' .IN 'x*y') + 1)
print(EVAL 'x')

--[[OUTPUT:
3
2       1
3       1
8
1
--]]

-- DavidManura, 2007-07

Hack: Ruby-like Symbols

Ruby has a feature called Symbols, which implements string-interning with a special syntax. There it's common practice to use symbols as the keys to hashes. As Lua already does string-interning by default, this hack is academic. It works by abusing the __index metamethod, filling with a function that just returns the key it was queried for as the value.

S=setmetatable({},{__index=function(S,k) return k end})
print( S.Symbol_Name )

A more useful expression of this hack would be as an alternative to any form of creating an enumeration table via a function. The string is cached for performance. Also, its been formatted for readability.

Enum = setmetatable( {}, {
  __index=function(E, k)
    E[k] = k
    return k
end})

print( Enum.Apple, Enum.Banana, Enum.Carrot )

Hack: Require module without quotes

Similiar to abusing the __index metamethod for ruby-style symbols, we can skip the need to enclose a module name in quotes when requiring it. This one builds up a string at the dot notation, as modules could be hidden deep in a directory tree. The arbitrary limit is that the directory structure assumes the first subdirectory. The require function needs to be wrapped as well, as it doesn't try to perform tostring on tables passed in.

do
  local wrap_require = require
  require = function(t) wrap_require(tostring(t)) end
  local mt = {}
  function mt:__index(key)
    return setmetatable({path=rawget(self, "path").."."..key}, mt)
  end
  function mt:__tostring()
    return rawget(self, "path") or ""
  end
  lib = setmetatable({path="lib"},mt)
end
do

require(lib.foo.bar)

Hack: s(g)etfenv in Lua 5.2

In Lua 5.2 both setfenv and getfenv were deprecated and removed. Although many prefer the dynamic enviorments, as opposed to lexical ones. The idea to this was that enviorments should not be accessible outside it's function. Though many of the hacks on this page use these. They are great for testing purposes, although not really reccomended in regular use, or as a major part of your program because it uses the debug library, which breaks some general Lua rules, and can be slow.

I did not write these, I found them both on the Internet and thought they deserved a place here, together. I think they were both posted on stackoverflow, a quick google search will reveal who wrote them. Here's the code:

--Setup
function setfenv(f, t)
   f = (type(f) == 'function' and f or debug.getinfo(f + 1, 'f').func)
   local name
   local up = 0
   repeat
      up = up + 1
      name = debug.getupvalue(f, up)
   until name == '_ENV' or name == nil
   if name then
      debug.upvaluejoin(f, up, function() return t end, 1)
   end
end

function getfenv(f)
   f = (type(f) == 'function' and f or debug.getinfo(f + 1, 'f').func)
   local name, val
   local up = 0
   repeat
      up = up + 1
      name, val = debug.getupvalue(f, up)
   until name == '_ENV' or name == nil
   return val
end

--Tests
function x()
   print(hi)
end

x() --nil
print(hi) --nil
setfenv(x, {print = print, hi = 5})
x() --5
print(hi) --nil

for i,v in pairs(getfenv(x)) do
   print(i, v)
end
--print function address
--hi 5
setfenv(x, {})

x() --attempt to call global 'print' (a nil value)