Some ways to get encapsulation in Lua

[Mark Hamburg <mhamburg@...>]

The following approaches to data encapsulation will work in the stock Lua
distribution. By encapsulation, I assume one wants to hand out a data object
and not have clients be able to interact with it except through method calls
or other published properties.


Function closures
-----------------
This is covered in Programming in Lua. Since a function can reference
upvalues and clients can't look inside without using the debug interface --
and in any context where encapsulation mattered, one should probably remove
the debug interface. The chief downside is that construction and usage don't
have much in common with other approaches to implementing objects in Lua.

    function makeAccount( bankID, balance )
        local self = function( msg, ... )
            if msg == "balance" then
                return balance
            elseif msg == "deposit" then
                local amount = select( 1, ... )
                assert( 0 <= amount )
                balance = balance + amount
            elseif msg == "withdraw" then
                local amount = select( 1, ... )
                assert( 0 <= amount )
                assert( amount <= balance )
                balance = balance - amount
            elseif msg == "payInterest" then
                local rate = select( 1, ... )
                assert( 0 <= rate )
                local interest = balance * rate
                assert( bankID == select( 2, ... )
                self( "deposit", interest )
            end
        end
        return self
    end

Usage is:

    account = makeAccount( secretBankID, 10000 )
    account( "withdraw", 500 )
    account( "payInterest", 0.05, secretBankID )
    account( "payInterest", 0.01, "scam" ) --> asserts out

The one change to Lua that would add sugar for this would be to specify that
obj:msg( ... ) translates to obj( "msg", ... ) if obj is a function. That's
just a change to OP_SELF.

Without this change, if one wants to make function-closure based
encapsulation look like other objects one needs to create a table containing
a reference to the function and have method declarations for the table that
do the appropriate conversion. For example:

    local AccountMeta = {}
    AccountMeta.__index = AccountMeta
    function AccountMeta:deposit( ... )
        return self.rep( "deposit", ... )
    end
    function AccountMeta:withdraw( ... )
        return self.rep( "withdraw", ... )
    end
    function AccountMeta:payInterest( ... )
        return self.rep( "payInterest", ... )
    end
    
    local function wrapAccountFunc( func )
        return setmetatable( { rep = func }, AccountMeta )
    end

Note that these functions could be generated on demand if we assumed that
all field access was to get methods.


Proxy tables with hidden representations
----------------------------------------
The basic idea here is that we hand out proxy tables while keeping the
representations hidden away. This requires a somewhat complicated set of
table relationships to implement:

1. The proxy needs to have a metatable that is unique to the proxy, is
protected from access by Lua code, and that references the representation.
This reference to the rep is necessary to make the link from proxy to
representation strong. We put it in the metatable so that we can hide it.

2. We need a fully-weak table mapping proxies to representations. This is
how a method invoked on the proxy will find it's representation since the
metatable protection will keep even code inside the encapsulation boundary
from getting the metatable. This table needs to be fully-weak and hence we
need the link in the metatable because just using a weak-keyed table and no
metatable link could fall victim to the cyclic issues with respect to weak
tables.

3. It is useful to maintain a fully-weak table mapping representations to
proxies so that we only create one proxy per representation. This could also
be done by having a link in the representation to its proxy.

Converting a representation into a proxy looks something like the following:

    local gRep2Proxy = setmetatable( {}, { __mode = "kv" } )
    local gProxy2Rep = setmetatable( {}, { __mode = "kv" } )

    function rep2proxy( rep )
        local proxy = gRep2Proxy[ rep ]
        if proxy then return proxy end
        local mt = {}
        mt.__metatable = "can't look inside me"
        mt.rep = rep
        mt.__index = < index function >
        mt.__newindex = < newindex function >
        proxy = setmetatable( {}, mt )
        gRep2Proxy[ rep ] = proxy
        gProxy2Rep[ proxy ] = rep
    end

We could probably standardize __index and __newindex functions in the proxy
so that they did the appropriate thing with respect to the encapsulated
object.

One could also build a system this way in which the reps themselves had the
necessary entries to be metatables for the proxies.

One might even be able to use function closures to eliminate the
proxy-to-rep table but this would come at the expense of instantiating even
more objects per usage.


Non-stock Lua
-------------
If we are prepared to change the Lua library, but not the VM or language,
then we can get encapsulation via private keys in tables. Private keys are
easy to generate: one just creates a table.


    local key_balance = {}
    local key_bankID = {}

    local AccountMeta = {}

    AccountMeta.__index = AccountMeta

    function AccountMeta:deposit( amount )
        assert( 0 <= amount )
        self[ key_balance ] = self[ key_balance ] + amount
    end

    function AccountMeta:withdraw( amount )
        assert( 0 <= amount )
        assert( amount <= self[ key_balance ] )
        self[ key_balance ] = self[ key_balance ] - amount
    end

    etc.

    function makeAccount( bankID, balance )
        return setmetatable( {
                [ key_balance ] = balance,
                [ key_bankID ] = bankID
            },
            AccountMeta )
    end

The catch is that these fields aren't really private as implemented since if
client code could obtain the keys, client code could access the fields
directly. How could client code obtain the keys? By iterating over the
table. So, we need a way to keep next and pairs from working on the table.
This could be done if pairs and next looked for metatable entries before
defaulting to the raw operations. The raw operations also have to be
inaccessible -- i.e., there can be no rawnext in the standard library.


Non-stock Lua 2: Less draconian metatable protection
----------------------------------------------------
If we could override metatable protection perhaps by allowing getmetatable
to take extra parameters it would pass to an unlock function in the
metatable, we could eliminate the weak tables in the proxy case. For
example, we could define getmetatable as follows using a hypothetical
rawgetmetatable:

    function getmetatable( t, ... )
        local mt = rawgetmetatable( t )
        if mt then
            local protect = mt.__metatable
            if type( protect ) == "function" then
                return protect( t, mt, ... )
            else
                return protect
            end
        end
        return mt
    end


Deeper Lua changes: Tables as their own proxies
-----------------------------------------------
We could reduce overhead in the proxy case if tables could be their own
proxies. A table that was its own proxy would need to be able to take two
forms as a TValue. In one form, it would be a table and would be just as
accessible as any other table. In another form, it would be marked as a
proxy and all access would go through metamethods much as if the table were
a userdata. I don't think this would actually be that hard to implement in
the VM. The consistency macro that makes sure the type in a TValue matches
the type in the heap-allocated struct itself would have to cope with
proxies, but that only generates code when the Lua core is built with
consistency checks active.

The extensions to the language spec would essentially be:

1. A new type value "proxy"

2. A new function toproxy which takes a table and returns the table as a
proxy.

Client code could use fully-weak tables to map back the other way or the
library could provide a metatable mechanism for proxies where the proxy's
metatable would provide a function determining whether to allow access to
the table based on additional parameters passed (e.g., an identity key). For
example:

    function mt:__fromproxy( proxy, table, key )
        if key == mySecretKey then
            return table
        else
            return nil
        end

This would be called by a library function fromproxy that would take a proxy
plus extra parameters to pass to the __fromproxy metamethod.
   
Mark