Meta Lua

lua-users home
wiki

Metalua is an alternative Lua compiler supporting compile-time metaprogramming and syntax extensions. It also contains a pure Lua library for parsing Lua source into ASTs.

Description

Metalua is backward compatible with Lua 5.1. You can dynamically add new idioms in the language, and provide metaprograms, i.e. programs which are run while the compiler is compiling another file and can read / modify /write / manipulate the program being compiled.

Usefulness

The practical interest is that you can extend the language, typically by importing a concept from other languages: you define a new syntax, and a manipulating function that turns this new idiom into a valid (although probably longer, harder to read and maintain) Lua program. Examples include proper support for various OO styles, ML-style structural pattern matching, design by contract, static or runtime type checking, lazy evaluation, list generation by comprehension...

Status

Metalua development is still in alpha stage).

Abstract syntax trees

The secret for efficient manipulation of any data is to represent it at the right abstraction level. Programs can be represented in many ways: as ASCII source files, token streams, abstract syntax trees, bytecode, compiled binary files... However, the consensus is that the most practical representation for automated program manipulation is the abstract syntax tree (AST). Since human users prefer plain text sources, Metalua lets you work with AST as well as plain text, mix them as you see fit, and offers facilities to translate from a format to another.

Learning curve

Writing extensions in metalua is not trivial: you need to know AST structure, to have a clear understanding of what metalevels are, and to be at ease shifting from one metalevel to another without getting confused. Obviously, mastering Lua is a must.

However, it's important to realize that if designing an extension is often not easy, using a well designed one should be easy.

This design is consistent with Lua's: it's easy, even for non-developers, to write basic programs. But if you want to do advanced things, you'll need to master non-trivial notions such as coroutines, environments, metatables, weak tables, higher order functions, etc. Rather than avoiding at all cost the need to look under the hood, Lua keeps it clean under that hood, so that advanced users can hack it with confidence. Notice that if it's not so easy to write, say, a proper API around a userdatum, using it should be be easy if it's properly designed.

This "shamelessly easier to use than write" approach is also similar to C++'s templates, although pretending that C++ is easy to use is a bit of a stretch: if you really think so, mastering metalua will really feel like a stroll in the park.

If you feel like metalua's idiosyncrasies are the hardest part of your extension design, it might mean that you're underestimating the complexity of implementing a correct extension, including the one you're writing.

Metalua vs. Lisp

Although Metalua technically allows to write programs with lots of small ad-hoc macros, as one often does in Lisp, it's not the intended purpose, and is not encouraged. Although macros can be very handy, especially in single coder projects developing everything from scratch, the price to pay in terms of readability and interoperability is often not worth it.

Lisp values freedom at all costs, and lets users write pretty much what they want without much guidance. It can become an issue if you ever have to work with some code not written by an outstanding hacker. At the other extreme, some extremely dull languages are a pain to work with, but grow a wealth of useful, robust and well maintained libraries. The aims with metalua are to:

Examples

Since Metalua manipulates a lot of trees, having pattern matching as in ML is very convinient. Such an extension is available in the language, and lets write things like below. Notice the presence of -{ extension 'foobar'} statements in the source: it loads extensions in the compiler, and by listing them you know which exact dialect of the language is used. You also don't need to remember how to compile each individual source file: this information is stored in it.

----------------------------------------------------
-- Lambda-Calculus evaluator (weak head normal form)
----------------------------------------------------
-{ extension "match" }

function replace(var, newval, term)
   match term with
   | `Var{ v } if v==var       -> return newval
   | `Var{ _ }                 -> return term
   | `Lambda{ v, _ } if v==var -> return term
   | `Lambda{ v, b }           -> return Lambda{ v, replace(var, newval, b) }
   | `Apply{ f, x }            -> return `Apply{ replace(var, newval, f), replace(var, newval, x) }
   end
end

function reduce_whnf(term)
   match term with
   | `Apply{ `Lambda { param, body }, arg } ->
      local x = replace (param, arg, body)
      return reduce_whnf(x)
   | _ -> return term
   end
end

What's interesting to see is how close the example above is from OCaml, the language from which the pattern matching idiom has been borrowed:

(* Type declaration: OCaml is a statically typed language! *)
type term =
| Var    of string
| Apply  of term * term
| Lambda of string * term

let rec replace var newval term =
   match term with
   | Var(v) when v==var       -> newval
   | Var(_)                   -> term
   | Lambda(v, _) when v==var -> term
   | Lambda(v, b)             -> Lambda(v, replace var newval b)
   | Apply(f, x)              -> Apply(replace var newval f, replace var newval x)

let rec reduce_whnf term =
   match term with
   | Apply(Lambda(param, body), arg) ->
     let x = replace param arg body in
     reduce_whnf x
   | _ -> term

A last example will show how an extension implementation looks like. We'll introduce the often begged for ternary choice operator, a.k.a. C's "?:" conditional. Since ":" is already used by Lua for method invocation, we'll make it "bool ? foo, bar" rather than "bool ? foo : bar". A correct (although inefficient) way to encode it is to put it in a function: "(function() if bool then return foo else return bar end end)()". This is implemented as:

local function t_builder(bool, suffix)
   local foo, bar = unpack (suffix)
   return +{ (function() if -{bool} then -{foo} else -{bar} end end)() }
end

mlp.expr.suffix:add{ "?", mlp.expr, "," mlp.expr, builder=t_builder }

(For the record, a more flexible and much more efficient implementation is available [with the compiler]).

Author

FabienFleutot

Related Projects that use Metalua

Links


RecentChanges · preferences
edit · history
Last edited October 31, 2009 6:42 pm GMT (diff)