Lua Fish

lua-users home
wiki

LuaFish provides various Lua modules for parsing Lua 5.1 source code via LuaPeg into abstract syntax trees (AST) and serializing ASTs back to Lua code. It also has experimental support for LISP-ish style macros, static type checking, and compiling Lua to C.

Description

LuaFish is similar in application to Metalua [1], Cheese [2], and LuaParse [3] but is based on LPeg [4]. This project might merge with Metalua in the future, though Metalua is based on gg. A very similar project is [Leg] [5].

The macro processing provides some interesting capabilities such as static type checking and code analysis. The macros operate at compile time on the AST to associate a type (or metatable) with a lexical. The macros use standard Lua function call syntax, so there is no change to the grammar but only to the semantics.

Status - WARNING

The source analysis parts of LuaFish were largely superseded by LuaInspect. LuaFish is still useful as an LuaPeg based parser though.

The parsing, AST manipulation, and serialization is fairly robust but could still have errors, and the interface is subject to change. The AST format should be brought in sync with the [Metalua AST format] (minus lineinfo which likely will be changed/removed in Metalua).

The macros, static type checking, and Lua->C compiler are incomplete or broken in various areas and should be considered experimental. In fact, they might no longer be maintained. See LuaInspect for newer stuff.

Please report and bugs or bug fixes to this wiki page.

Download files

Author

DavidManura

Examples

Example: Converting Lua to an AST

The AST of the following file

-- example for Lua->C compiler.
local x,y = 4,5
x = x + 1
local function f(x)
  return x * x
end
x = f(x)
print(x)

can be quickly displayed as follows:

$ lua bin/luafish.lua -a examples/1.lua
{tag="Block",{tag="Local",{tag="NameList",{tag="Id","x"},{tag="Id","y"}},{tag="E
xpList",{tag="Number",4},{tag="Number",5} } },{tag="Assign",{tag="VarList",{tag="I
d","x"} },{tag="ExpList",{tag="Op","+",{tag="Id","x"},{tag="Number",1} } } },{tag="L
ocalFunctionDef",{tag="Id","f"},{tag="NameList",{tag="Id","x"}},{tag="Block",{ta
g="Return",{tag="ExpList",{tag="Op","*",{tag="Id","x"},{tag="Id","x"} } } } }  },{tag=
"Assign",{tag="VarList",{tag="Id","x"}},{tag="ExpList",{tag="Call",{tag="Id","f"
},{tag="ExpList",{tag="Id","x"} } } } },{tag="Call",{tag="Id","print"},{tag="ExpList
",{tag="Id","x"} } } }

Example: Static Type Checking and Undefined Variable Detection

This example demonstrates some of the static type checking capabilities. Types are bound to lexicals at compile time via the TYPED/NUMBER/STRING macros or or by automatic deduction. Types are checked and constant expressions are evaluated at compile time. The REQUIRE macro provides compile-time macro and type import and also does a regular require at run-time. All global variables are disabled via the NOGLOBALS macro (except those bound to lexicals via REQUIRE), so global variable access (including misspelled lexicals) trigger a compile-time error.

-- type_usage2.lua
-- LuaFish static type checking example.
-- Using math library.
-- Requires LuaFish 0.4.
--
-- Note: CAPS identifiers are typically macros.

-- Compile-time import of static typing macros NUMBER, STRING, and TYPED
REQUIRE 'luafish.type'

-- disable global variable usage
NOGLOBALS()

-- Compile-time import of static type definitions for standard modules.
local math = REQUIRE 'math'
local _G = REQUIRE '_G'

local print = _G.print

-- False conditional demonstrates that static type checking is done
-- at compile-time.
if false then
  print(math.sqrt) -- ok
  --print(math.asdf) -- compile error: asdf not in math

  --print(math.sqrt('one')) -- compile error: arg must be number
  -- print(math.sqrt(2,3)) -- compile error: num args
  -- print(math.sqrt(-1)) -- compile error: arg must be non-negative
  print(math.sqrt(2)) -- ok

  local x = 2  -- weak, implicit type Number
  --x() -- compiler error: not callable
  x = print() -- implicit type not unknown after calling unknown function
  x() -- ok now

  -- Note: compare this to the above.
  local x = TYPED(-3) -- bind strong, implicit type to lexical
  --local x = NUMBER(-3)  -- alternate form with explicit type
  --x() -- compile error: not callable
  x = print() -- does not modify strong type
  --x() -- compile error: not callable

  local x = -3
  --print(math.sqrt(x)) -- compile error: arg must be non-negative
  x = x + 2
  --print(math.sqrt(x)) -- compile error: arg must be non-negative
  x = x + 1
  print(math.sqrt(x)) -- ok

  --math.sqrt(math.sin(-math.pi/2)) -- compile error: arg must be non-negative

  local x = STRING(print()) -- bind string type, unknown value f()
  x = 5 -- doesn't affect strong type
        -- TODO: we could guard against such assignment.
  --print(math.sqrt(x)) -- compile error: arg must be number

  local sqrt = math.sqrt
  -- print(sqrt(-2)) -- compile error: arg must be non-negative

  local sqrt = TYPED(math.sqrt)
  -- print(sqrt(-2)) -- compile error: arg must be non-negative
end

print 'type_usage2.lua : done'

Example: Macro Handling

Here is another example of using the experimental macro processing capabilities with modules. This uses one of the styles for processing macros.

-- module_usage2.lua
-- LuaFish example that tests square2.lua module.
-- It uses both the static type and runtime definition
-- in square2.lua.

print 'DEBUG:main:begin compiletime' -- trace

-- TSquare is the static type of square2.
local TSquare = require "square2"

-- This compiles and executes the given code string.
-- During compilation, the SQUARE macro is evaluated.
-- The SQUARE macro is defined as TSquare.bind, which
-- binds the given lexical to the TSquare static type
-- and returns an empty code block that replaces the macro
-- in the AST.  The code is then statically checked
-- against the bound static types.  Finally, the code
-- is executed.
require "luafish.staticmodule" {
  SQUARE = TSquare.bind, ISSQUARE = TSquare.isa
} [[
  print 'DEBUG:main:end compiletime'
  print 'DEBUG:main:begin runtime'

  -- Load run-time behavior of square2.
  local Square = require "square2" . class

  -- Create instance.  Assign to lexical.
  -- Bind static-type to lexical.
  local m = Square.create(5); SQUARE(m)

  -- This demonstrates that even though the following code is
  -- not executed at run-time, it is still compile-time checked.
  if false then
    m:setcolor('blue') -- ok
    local a = m.hello           -- compile error (field name)
    local b = m.setcolor(m,'blue') -- ok
    local b = m.setcolor(m,5)   -- compile error (arg type)
    local b = m.setcolor(m,5,6) -- compile error (num args)

    local b = (m * 2):area(1)   -- compile error (num args)

    -- local a = false + false  -- compile error (op not defined)
    local a = false and true    -- ok
    local a = 5 + 3^3           -- ok
  end

  print 'DEBUG:main:end runtime'
]]

--[[OUTPUT:
DEBUG:main:begin compiletime
DEBUG:square2:begin compiletime
DEBUG:square2:end compiletime
DEBUG:square2:begin runtime
DEBUG:square2:end runtime
static __index  [TSquare Class] setcolor
static call     {"Id","m"}      {"String","blue"}
static __index  [TSquare Class] hello
ERROR:  hello not in [TSquare Class]
static __index  [TSquare Class] setcolor
static call     {"Id","m"}      {"String","blue"}
static __index  [TSquare Class] setcolor
static call     {"Id","m"}      {"Number",5}
ERROR:  second param must be string
static __index  [TSquare Class] setcolor
static call     {"Id","m"}      {"Number",5}    {"Number",6}
ERROR:  second param must be string
ERROR:  expected two arguments
static __mul    [TSquare Class] table: 0127EE68
ERROR:  first op must be TSquare
static __index  [TSquare Class] area
static call     {"Parens",{"*",{"Id","m"},{"Number",2} } }        {"Number",1}
ERROR:  expected zero arguments
DEBUG:main:end compiletime
DEBUG:main:begin runtime
DEBUG:main:end runtime
--]]

where the module is defined as

-- square2.lua
-- LuaFish example of a module that indirectly
-- contains macros.  Contains both
-- static type check and run-time behavior.

-- Static type definition.
local TSquare = {}; do
  print 'DEBUG:square2:begin compiletime'  -- trace

  local Macro = require "luafish.macro"

  -- Helper functions.
  local report = function(...) print('ERROR:', ...) end
  local check = function(test,message)
  if not test then report(message) else return true end
  end

  setmetatable(TSquare, {
    __tostring = function() return '[TSquare Class]' end
  })
  -- bind lexical to this type.
  function TSquare.bind(obj_ast)
    obj_ast.stype = TSquare
  end
  -- tests if expression is of this type
  function TSquare.isa(obj_ast)
    return 'value', obj_ast.stype == TSquare
  end
  local is_method = {area=true,perimeter=true,setcolor=true}
  function TSquare:__index(k)
    print('static __index', self, k)
    if not is_method[k] then
      report(tostring(k) .. ' not in ' .. tostring(TSquare))
    end
    if k == 'setcolor' then
      return function(self, o, ...)
        print('static call', self, o, ...)
        check(self.stype == TSquare, 'first param must be TSquare')
        check(Macro.TString.isa(o.stype), 'second param must be string')
        if select('#', ...) ~= 0 then
          report('expected two arguments')
        end
      end
    else
      return function(self, ...)
        print('static call', self, ...)
        if select('#', ...) ~= 0 then
          report('expected zero arguments')
        end
      end
    end
  end
  function TSquare:__mul(other)
    print('static __mul', self, other)
    if not (check(stype == TSquare, 'first op must be TSquare') or
            check(Macro.TNumber.isa(other), 'second op must be number'))
    then return end
    return TSquare
  end
  print 'DEBUG:square2:end compiletime'
end

-- Run-time behavior.
TSquare.class = require "luafish.staticmodule" {} [[
  print 'DEBUG:square2:begin runtime'

  local Square = {}
  Square.__index = Square
  function Square.create(length)
    return setmetatable({length=length}, Square)
  end
  function Square:area(length) return self.length^2 end
  function Square:perimeter(length) return self.length*4 end
  function Square:setcolor(color) self.color = color end
  function Square:__mul(other, val)
    return Square.create(self.length * val)
  end

  print 'DEBUG:square2:end runtime'

  return Square
]]

return TSquare

You can think of the static type description as a metatable that is attached to a lexical and operated on at compile time.

Another way to use macros is to place the macro-enabled code in a separate file and use the replacement macro-enabled versions of require or dofile.

Example: Lua -> C Compiler

This Lua->C compiler is very-very preliminary and makes many assumptions. It's more of a prototype. It should have more checks and trigger errors if it cannot ensure valid compilation to equivalent C.

$ lua 1.lua 
25 
 
$ lua lib/luafish/lua2c.lua 1.lua | gcc -xc - 
 
$ ./a.out 
25.000000 
 
 
-- input: 1.lua -- 
 
local x,y = 4,5 
 
x = x + 1 
 
local function f(x) 
  return x * x 
end 
 
x = f(x) 
 
print(x) 
 
 
-- output: 1.c -- 
 
#include <stdio.h>
double f(double x) {
return x * x;
}
int main() {
double x = 4;
double y = 5;
x = x + 1;
x = f(x);
printf("%f\n", x);
return 0;
}

For more examples and details, see the distribution examples and source code.

See Also

LuaFish Source Analysis - Take Two (a.k.a. LuaAnalyze)

Here's a preview of a redesigned source analyzer based on some principles learned from the LuaFish work (warning: alpha version): [luaanalyze-20080925b.tar.gz]

The new code tries to make the design more practical. It also uses gg/mlp (from Metalua) rather than LPeg. Example file:

-- examples/ex1.lua
do
  --! typeimport('luaanalyze.library.standard')
  --! typematch('idx$', 'luaanalyze.type.number')
  --! checkglobals()
  --! checktypes()

  for my_idx=1,10 do
    local s = string
    local f = s.format
    --print(f(my_idx)) -- fails: got number expected string

    --print(myy_idx) -- fails: undefined global
  end
end

print(myy_idx) -- no error

To check, run "lua luaanalyze.lua examples/ex1.lua". Comments prefixed by '!' are interpreted by the source analyzer. There's quite a few interesting things that should be stated about the above example (more on this later).

WARNING: luaanalyze is superseded by LuaInspect.

See Also


RecentChanges · preferences
edit · history
Last edited September 22, 2010 5:28 am GMT (diff)