Re: Alternative (better?) __index implementation

[Rici Lake <lua@...>]

On 28-Nov-07, at 12:25 PM, alex.mania@iinet.net.au wrote:

> It seems correct when I think about it, because I cannot envision a 
> scenario where
> you have:
>
> Table1.__index = Table2; Table2.__index = function()
>
> Yet would prefer Table2 do be passed to the function then Table1, 
> should Table1 be
> the object being indexed.

Suppose Table2 were created with Memoize:

function Memoize(func)
  return setmetatable({}, {
    __index = function(self, key)
                local rv = func(key)
                self[key] = rv
                return rv
              end,
    __call = function(self, key) return self[key] end
  }
end

This is going to act very oddly if self is wrong. (In particular,
the __call metamethod will fail.)

Here's another real-life example. While Memoize above relies on closures
to work, it can sometimes be useful to stash information (like func, in
the above case) in the proxy table itself. This is possible without 
leaking
any information if the key(s) being used are unavailable to the client.
In particular, the proxy table itself is unique and unexposed (if the
metatable is locked, it is completely unexposed.)

So, for example, one could rewrite Memoize as follows:

do
  local meta = {__metatable = {}}
  function meta:__index(key)
    local rv = self[self](key)
    self[key] = rv
    return rv
  end
  function meta:__call(key)
    return self[key]
  end
  function Memoize(func)
    local t = {}
    t[t] = func
    return setmetatable(t, meta)
  end
end

Although that's slightly less readable, it's a lot more efficient; the 
first version
creates two closures, one upvalue, and two tables for every Memoized 
function, while
the second version only creates a single table, with one key-value 
pair. Clearly,
I could have put more information than just a function in the sentinel 
key-value
mapping, had I needed to.

Another way to safely associate private data with proxy tables, also in 
common
use, is to use the proxy table as the key in a one or more weak-keyed 
mappings.

>  And in any case, being passed Table2 destroys any
> knowledge of Table1, but the other way around no information is lost 
> as Table2 can
> be found through calls to getmetatable.

Yes, but not very easily. You'd have to simulate the action of 
traversing the
proxy tables, and that would fail if any of the metatables were locked 
with
a __metatable key. So how would you get version 2 of Memoize to work?

The basic problem with special-casing the __index chaining is that it 
becomes
non-composable. The behaviour of a functable changes if it is the 
target of
an __index metamethod, in a way which is hard to predict. The putative 
savings
(one function call) are just not worth the breaking of orthogonality.

One backwards compatible extension which might solve some issues (such 
as
the "redundant" function call in your string example) would be to add a 
__proxy
metakey, and change the semantics of the get event to:

function get(self, key)
  local rv = rawget(self, key)
  if rv == nil and getmetatable(self).__proxy then
    rv = getmetatable(self).__proxy[key]
  end
  if rv == nil and getmetatable(self).__index then
    -- to be entirely backwards compatible, we should
    -- try using __index as a table first. But we'd drop
    -- that eventually
    rv = getmetatable(self).__index(self, key)
  end
  return rv
end

Aside from being slightly more efficient in the not uncommon
case that you have a proxy table and a metafunction, and only
want to use the metafunction if the key is not in the proxy table,
this has the advantage that it clearly separates the two operations:
 *indexing* the __proxy and *calling* the __index. This would make
it possible to use, for example, userdata with __index metamethods
as __proxy metavalues, and tables with __call metamethods as
__index metavalues, neither of which are currently possible;
consequently, it would improve orthogonality (imho).

Rici