Classes As Constructor Closures

lua-users home
wiki

Introduction

In ten years or more of Lua programming, I have experimented with numerous approaches to Object Orientation (OO) from "Programming in Lua", "Lua Programming Gems" and this wiki as well as schemes of my own, some with extensive 'C' support libraries. I have eventually standardised on the scheme outlined in this paper which is very flexible and relatively simple, can be implemented entirely in Lua and possesses some desirable OO characteristics that I've not seen in other Lua implementations.

Desirable OO Characteristics

It is commonly agreed that an OO system should provide encapsulation of multiple values (fields) and functions operating on those fields (methods). It is fairly commonly agreed that the system should provide an easy way of generating multiple objects sharing characteristics (most commonly method implementations). Classes or prototypes are two ways of implementing this.

I also want my classes to be seamless extensions of the Lua type system. In a pure OO language, there would be no distinction between types and classes, but a good compromise for Lua is for classes to be types but not vice versa. Given an unknown Lua value, we will need a function which can determine if it is of a specified class or type.

Another desirable characteristic is inter-working between classes and objects created in 'C' using the Lua API and those created in pure Lua. This allows 'C' libraries to provide classes and functions or methods defined in those libraries to create and return objects. It should ideally also be possible for 'C' code to create objects from classes defined in Lua.

Classes implemented as Constructor Closures

The crux of the scheme I propose in this paper is that classes are implemented as Lua function closures that return a new object of the class when executed. All the resources required to create an object are stored in upvalues of the class closure or passed as parameters to the closure function. This means that classes are first-class Lua values independent of their storage.

Some characterised upvalues must be present in all classes to support type matching. This is referred to as the TID, for Type ID. It must also be possible to derive a TID value from objects generated by the classes so that a generic predicate function can match an object with its class. By the simple expedient of defining the TID of values which are not classes or objects to be the Lua type name, this scheme can be extended so all Lua values have a TID and can be type-matched using the same predicate function.

A nice consequence of this scheme is a constructor syntax for objects that represent collections. For example, assuming the definition of a class called List, we can write:


mylist = List{"ham", "eggs", "toast"}
        

This uses a standard Lua syntax shortcut to allow the function call braces to be omitted. Within the List function, we can detect that one parameter has been passed, a table with no metatable (call this a 'rawtable'), and in this case convert it in place into an object without having to copy the entries. I think this seems like a very natural extension of the Lua table constructor syntax. Alternative construction syntaxes can also, or alternatively, be provided using other parameter signatures.

Creating Classes and Objects

Having been quite prescriptive in defining a class implementation, we can be much more flexible about the object implementation whilst still preserving universal type matching. In this section, I will show that the major object patterns suggested in the literature can all be accommodated within this scheme.

Firstly, I will demonstrate a class that produces metatable-based objects. In this pattern, the object is a Lua table (or possibly a userdata if implemented in 'C') with a metatable common to all objects of the same class. The methods and metamethods of the class are stored in this metatable while the fields are stored in the object table. The metatable is also the class TID.


do  -- Class 'MetaBaseClass'
  local _TID = class.newmeta()

  -- Class Closure:
  local _C = function(p1)
    local o
    if class.istype(p1, 'rawtable') then o = p1 else o = {} end
    o.value = o.value or 0
    setmetatable(o, _TID)
    return o
  end

  -- Methods:
  function _TID:mymethod()
    class.checkmethod(self, _C)
    print("Executing 'mymethod'")
  end

  -- Metamethods:
  function _TID.__add(p1, p2)
    class.checkmethod(p1, _C)
    class.checkmethod(p2, _C)
    local rv = _C()
    rv.value = p1.value + p2.value
    print("Executing add metamethod")
    return rv
  end
  
  -- Class Closure is a first-class value, for example, store it as a global:
  MetaBaseClass = _C
end -- Class 'MetaBaseClass'

The function class.newmeta() is a simple helper function that creates a new metatable and sets the index metamethod to self-refer. class.istype() is a multi-purpose type testing predicate and class.checkmethod() wraps this test and raises an error if the predicate is false (I will show an implementation of these functions later in this paper). Notice that routine use of class.checkmethod also means that all methods of a class hold upvalue references to the class itself facilitating creation of return parameters of the same class.

To create objects, just call the class:


obj1 = MetaBaseClass{value = 13}
obj2 = MetaBaseClass{value = 10}
obj1:mymethod()
obj3 = obj1 + obj2
print(obj3.value, 23)

To support single inheritance, class.newmeta() can be extended to take a parent class. This gets the parent TID (which is also its metatable) and sets it as the metatable of the new metatable (metametatable?). The parent class is also passed through unchanged as a syntax convenience:

        
do  -- Class 'MetaChildClass' (Inherits MetaBaseClass)
  local _TID, _PC = class.newmeta(MetaBaseClass)

  -- Class Closure:
  local _C = function(p1)
    local o = _PC(p1)
    -- Extra initialisation for child class
    setmetatable(o, _TID)
    return o
  end

  -- Methods are inherited automatically, but may be overridden.

  -- To inherit metamethods add explicit delegations:
  _TID.__add = class.gettid(_PC).__add

  MetaChildClass = _C
end -- Class 'MetaChildClass'

obj4 = MetaChildClass{value = 2}
obj4:mymethod()
print((obj4 + obj1).value, 15)

The second pattern I will explore is prototype based. Classically, prototype approaches do not use the concept of 'class' at all; instead any object may act as a prototype for creating further objects. However I will use a standard class 'wrapper' around the prototype object and this will implement the table cloning code. The objects produced are Lua tables which contain the methods as well as the fields. If metamethods are required, a metatable will also be needed. To support type matching, a metatable will always be assigned even if it is empty.


do  -- Class 'ProtoBaseClass'
  local _C, _PRT, _MTB = class.newproto()

  _PRT.field = "Hello from ProtoBaseClass"

  function _PRT:method()
    class.checkmethod(self, _C)
    print(self.field)
  end

  ProtoBaseClass = _C
end -- Class 'ProtoBaseClass'        
        

The support function class.newproto() does most of the work in this case. It generates the class (function closure) and also returns references to the prototype table and the metatable ready to be populated. The class function is standardised, providing the functionality to clone the prototype table.


obj5 = ProtoBaseClass()
obj5:method() -- Prints 'Hello from ProtoBaseClass'
obj6 = ProtoBaseClass{field="Hello from obj6"}
obj6:method() -- Prints 'Hello from obj6'
        

The standard class function takes an optional table parameter which can override (initialise) existing fields after they have been copied from the prototype.

This pattern can accommodate multiple inheritance (aggregation). The function class.newproto() can accept any number of object (table) parameters. These tables, and their metatables are merged to create the prototype table and metatable. Merge is in parameter order, so fields in later parameters override those with the same name in earlier parameters.


do  -- Class 'ProtoAggregateClass'
  local _C, _PRT = class.newproto(ProtoBaseClass())
  _PRT.field = "Hello from ProtoAggregateClass"
  ProtoAggregateClass = _C
end -- Class 'ProtoAggregateClass'        
        
obj7 = ProtoAggregateClass()
obj7:method() -- Prints 'Hello from ProtoAggregateClass'

Note that in this pattern, inheritance is from an object of the parent class, not from the class itself.

The final pattern I will consider is the single function pattern. This has an object implementation similar to the class implementation (indeed a class is also an object of this type). The object has just one method which forms a closure with the fields stored as upvalues. This pattern does not support inheritance or metamethods.


do -- Class 'FunctionClass'
  local _TID = "FunctionClass"
  local _TID_O = _TID
  local _C = function()
    local tid = _TID
    local val = 0
    return function()
      local tid = _TID_O
      val = val + 1
      return val
    end
  end
  FunctionClass = _C
end -- Class 'FunctionClass'

obj8 = FunctionClass()
obj9 = FunctionClass()
print(obj8(), 1)
print(obj8(), 2)
print(obj9(), 1)

In this pattern the TID is only used for type testing and is set as an arbitrary, but unique, string. Alternatively an empty table could be used. Because the implementation of the class and the object cannot be distinguished by Lua type, an extended tag convention is used. The tag is exactly "_TID" for the class, but has a suffix when used in the object. By this convention, a class is an object, but an object is not a class.

Support Functions

The TID can be derived from any Lua value using the following rules applied in order until a TID value is determined:

1. The TID is the non-nil value of any metamethod with the key '__tid'.

2. If the value is a table or userdata with a metatable, then the TID is the metatable.

3. If the value is a function with an upvalue with a name starting "_TID" (the tag), then that upvalue is the TID. This applies to function closures created in Lua.

4. If the value is a function with at least two unnamed upvalues, the first of which has a string value starting "_TID" (the tag), then the second upvalue is the TID. This applies to function closures created in 'C'.

5. Otherwise the Lua type of the value is the TID (which will be a string).

A class is a value with a TID determined by rule 3 or 4 with the tag being exactly "_TID". If the tag has any additional characters after "_TID" the value is an object, but not a class.

The following is a gettid implementation in pure Lua:


do
  local getupvalue = require('debug').getupvalue
  class = class or {}
  class.gettid = function(v)
    -- Rule 1: metafield
    local mt = getmetatable(v)
    if mt and mt.__tid then return mt.__tid, "object_mf" end
    -- Rule 2: metatable
    if mt and (type(v) == 'userdata' or type(v) == 'table') then 
      return mt, "object_mt"
    elseif type(v) == 'function' then
      local un, uv = getupvalue(v, 1)
      local r = "class"
      if un and un ~= "" then
    -- Rule 3: named upvalue
        local i = 2
        while un and un:sub(1,4) ~= "_TID" do
          un, uv = getupvalue(v, i)
          i = i + 1
        end
        if un and #un ~= 4 then r = "object_fn" end
      elseif un and type(uv) == 'string' and uv:sub(1,4) == "_TID" then
    -- Rule 4: upvalue pair with tag
        if #uv ~= 4 then r = "object_fn" end
        un, uv = getupvalue(v, 2)
      end
      if un then return uv, r end
    end
    -- Rule 5: Lua type name
    return type(v), "type"
  end
end

Having obtained the TID of the type (or class) and that of the value (or object), identity is determined by the following rules:

1. If the TID values are Lua equal functions, then that function is called passing it the value and the type. The result of the test is the Boolean value of the first return.

2. If the TID of the value is a table which has an entry with a key number 1, then the test is true if the TID of the type matches any entry keyed with a consecutive integer from 1 up.

3. If the TID of the value is a metatable and that of the type a table, then the test is true if the type TID matches the metatable, the metatable of the metatable and so on until a metatable has no metatable.

4. If none of the above applies, the test is a simple Lua equality test between the two TID values.

The following Lua function implements these matching rules and also incorporates standard Lua type detection using a string in place of the type parameter, some additional useful type predicates and the ability to determine if two arbitrary Lua values (except strings) are of the same type:


class.istype = function(vl, ty)
  local tvl = type(vl)
  local tid1, isc = class.gettid(vl)
  if type(ty) == 'string' and ty ~= "" then
    if ty == "class" then
      return isc == 'class'
    elseif ty == "object" then
      return isc ~= 'type'
    elseif ty == "rawtable" then
      if tvl ~= "table" then return false end
      return getmetatable(vl) == nil
    elseif ty == "callable" then
      if tvl == "function" then return true end
      local m = getmetatable(vl)
      if not m then return false end
      return type(m.__call) == "function"
    else
      return tvl == ty
    end
  end
  local tid2 = class.gettid(ty)
  if tid2 == tvl then return true end
  if type(tid1) == 'function' and tid1 == tid2 then return not not (tid1(vl, ty)) end
  if type(tid1) == 'table' and tid1[1] then
    for i=1, #tid1 do
      if tid2 == tid1[i] then return true end
    end
  end
  if isc == 'object_mt' and type(tid2) == 'table' then
    repeat
      if tid2 == tid1 then return true end
      tid1 = getmetatable(tid1)
    until not tid1
  end
  return tid1 == tid2
end

class.checkmethod = function(vl, ty)
  if not class.istype(vl, ty) then error("Bad method call") end
end

-- Export "istype" as a global since it will be widely used:
istype = class.istype

The support function for the metatable pattern is now straightforward:


class.newmeta = function(pcl)
  local mt = {}
  if pcl then
    if not class.istype(pcl,'class') then error("Bad Parent Class") end
    local pmt = class.gettid(pcl)
    if not class.istype(pmt, 'table') then error("Bad Parent Class") end
    setmetatable(mt, pmt)
  else
    setmetatable(mt, nil)
  end
  rawset(mt, "__index", mt)
  return mt, pcl
end

The support function for the prototype pattern is more subtle. The outer function merges the parent objects into a new class prototype table and metatable. This metatable is also the TID for the class. The metatables of the parent objects are also referenced under numeric keys in the new metatable, which allows the istype function to match objects of this class with objects of any of its parent classes, or classes enclosing these objects. The inner function returned in the class closure copies the prototype upvalue into a new object table, performs a final merge using any initialisation table, and sets the TID upvalue as the metatable for the new object.


class.newproto = function(...)
  local _TID, _PRT, pmt = {}, {}, nil
  for i, t in ipairs{...} do
    if type(t) ~= 'table' then error("prototype must be table") end
    pmt = getmetatable(t)
    for k, v in pairs(t) do _PRT[k] = v end
    if pmt then 
      for k, v in pairs(pmt) do _TID[k] = v end
      _TID[#_TID + 1] = pmt
    end
  end
  local _C = function(init)
    local tid, prt, ob = _TID, _PRT, {}
    for k, v in pairs(prt) do ob[k] = v end
    if init then for k, v in pairs(init) do
      if ob[k] == nil then
        error("attempt to initialise non-existant field: " .. k)
      end
      if type(k) ~= 'string' or k:sub(1,1) == '_' then
        error("attempt to initialise private field: " .. k)
      end
      ob[k] = v
    end end
    setmetatable(ob, tid)
    return ob
  end
  return _C, _PRT, _TID
end
                

Examples of Type Testing

Using the example classes developed earlier, the following type tests all print 'true':


print( istype(MetaBaseClass(), MetaBaseClass) )
print( istype(MetaChildClass(), MetaBaseClass) )
print( istype(MetaChildClass(), MetaChildClass) )
print( not istype(MetaBaseClass(), MetaChildClass) )
print( istype(MetaChildClass(), MetaBaseClass() ) )
print( not istype(MetaBaseClass(), MetaChildClass() ) )
print( istype(MetaBaseClass(), 'table') )
print( not istype(MetaBaseClass(), 'rawtable') )
print( istype({}, 'rawtable') )
print( istype(ProtoBaseClass(), ProtoBaseClass) )
print( not istype(ProtoAggregateClass(), MetaBaseClass) )
print( istype(ProtoAggregateClass(), ProtoAggregateClass) )
print( istype(ProtoAggregateClass(), ProtoBaseClass) )
print( istype(FunctionClass(), FunctionClass) )
print( istype(FunctionClass(), 'function') )
print( not istype(FunctionClass(), 'table') )
print( istype(FunctionClass(), 'callable') )
print( istype(FunctionClass(), 'object') )
print( not istype(FunctionClass(), 'class') )
print( istype(FunctionClass, 'class') )
print( istype(FunctionClass, 'object') )
print( istype(12, 'number') )

Interworking with 'C'

For an example of a hybrid implementation of these ideas partly in 'C' and partly in Lua, check out the files LibClass.h; LibClass.cpp and LibClass.lua here:

https://github.com/JohnHind/Winsh.lua/tree/master/Winsh/Libraries

In particular, the implementation of the List class in these files starts off in 'C' and is subsequently completed in Lua!

Conclusion

By being prescriptive about the implementation of the concept of 'class', and by implementing a highly flexible type matching architecture, we can afford to be much looser about the implementation of objects. Different classes in the same Lua state can implement their objects in many different ways and still maintain a homogeneous type system in which the type of any value can be matched and characterised by a generic predicate function.

Implementing classes as function closures such that the function enclosed is the factory function or constructor for objects of that class brings big advantages. Notably classes defined in this way can be self contained and first class Lua values which do not depend on registry or global resources and are independent of storage. This approach also leverages the syntax shortcut provided for named function parameters to provide a syntax for collection class constructors which seems like a natural extension of Lua table constructors.

JohnHind (13 February 2014)


RecentChanges · preferences
edit · history
Last edited February 24, 2014 11:03 am GMT (diff)