Generalized pairs, was Re: linking tables to C++ objects?

[Rici Lake <lua@...>]

On 26-Feb-07, at 7:31 PM, Graham Wakefield wrote:

> Yes, I've been trying this approach today; mapping __index and 
> __newindex metamethods of the userdata to the userdata environment 
> table. A disadvantage of this is that in Lua my object behaves as a 
> table for general use, but not as a table for functions such as 
> table.foreach() and generic for, which could be confusing for end 
> users.  Is there a way to implement a next() method for the generic 
> for construction?

Actually, you want to implement pairs(), not next(), for the obvious 
reason that you will always use the same next() method on a given 
object, so you might as well just look it up once at the beginning of 
the loop rather than on every iteration. There are other good reasons 
to think
of polymorphic pairs() rather than next(), but I won't go on about it
here -- the main one is that it's usually a lot easier.

I have a small module, which needs some work to be honest, lurking
on the wiki at http://lua-users.org/wiki/GeneralizedPairsAndIpairs
(after a wiki reorganization, it's no longer at the top of that
page, but it's still there), which effectively redefines pairs()
to consult a __pairs metamethod in order to create the iterator
triple.

You can do this pretty easily in Lua (there's a one-liner on that
wiki page); here's a fairly complete implementation:

do
  local _next = next
  -- Unlike the "real" pairs, this one lets you say
  -- where to start (default at the beginning)
  local function _pairs(obj, state)
    return _next, obj, state
  end
  rawpairs = pairs -- for posterity
  function pairs(obj, ...)
    local m = getmetatable(obj)
    return (m and m.__pairs or _pairs)(obj, ...)
  end
  -- Now, we can define next() in terms of pairs(),
  -- for people who insist on using next():
  function next(obj, ...)
    -- __pairs better take a starting point, like the
    -- redefinition above. First we get the iterator triple:
    local func, obj, state = pairs(obj, ...)
    -- Then we call it once to advance to the next state, value(s):
    return func(obj, state)
  end
  -- Finally, we can reimplement table.foreach() (which is, in
  -- any event, deprecated). We need an auxiliary function to
  -- "hold onto" the possible multiple values.
  -- This function takes the function to iterate, and all the
  -- returns of the iterator (the first one is the new state);
  -- it returns the return value in case the function breaks
  -- the loop, and the next state:
  local function aux(func, s, ...)
    return func(s, ...), s
  end
  -- We can't just do a 'for' loop, because we don't know
  -- how many returns we'll get. So we just do it by hand:
  function table.foreach(obj, func)
    -- get the iterator triple
    local f, o, s = pairs(obj)
    local rv
    repeat rv, s = aux(func, f(o, s))
    until rv ~= nil or s == nil
    return rv
  end
end

The main issue with implementing this in Lua is not efficiency
(even table.foreach isn't bad). The problem is that the metatable
might (quite reasonably) be locked with a __metatable key, and
that would prevent us from actually getting at the __pairs
pseudo-metamethod. Implementing it in C avoids that problem.