tables with __index metamethods and next()

["Dimitris Papavasiliou" <dimitris@...>]

Hi all,

I hope this hasn't come up already, but an archive search didn't
reveal anything, so:  Suppose you have the following table:

foo = {
   [1] = 1,
   [2] = 2,
   [3] = 3
}

and with the following __index metamethod:

function (t, k)
   if k == "sum" then
      local sum = 0

      for _, v in ipairs(t) do
         sum = sum + i
      end

      return sum
   end
end

So that you can use print (foo.sum) to get the sum of the elements.  A
"product" key could be likewise defined.  Now the problem is that
although these look just like normal keys they won't show up when
iterating through the table with next() or with pairs() and ipairs()
of course.  I'm using lua to interface with low-level code in C so
that I have many tables that use such metamethods to perform low-level
actions when certain keys are written to or read.  It would be nice if
I could treat these just like normal tables without having to keep in
mind which values are actual keys and which are handled via
metamethods.

So what do you guys think about adding a __next metamethod for
example?  This could either replace  the next() function completely
when present or it could be called in addition to the real next(). 
The first method would probably be preferable as it is more general. 
This way one could have a __next metamethod like the following to
solve this issue for the previous table:

function (t, k)
  local extrakeys = {sum = true, product = true}
  local static

  -- Look for a statically defined key first (like foo[1])

  statickey, staticvalue = next (t, k)

  if statickey ~= nil then
     return static, staticvalue
  else
     local nextkey

     -- Otherwise check whether this is one
     -- of the "extra" keys.

     if extrakeys[k] == nil then
        nextkey = next (extrakeys)
     else
        nextkey = next (extrakey, k)
     end

     -- And if so call the __index metamethod
     -- to retrieve its value and return it
     -- along with the key.

     return nextkey, t[nextkey]
   end
end

I wrote this function without trying it obviously but it's there for
illustration purposes only.  It could probably be written in a more
elegant way.  Also other solutions would be possible, like, for
example, allowing to specify extra keys in the metatable
(foometa.__extrakeys = {"sum", "product"}) and having next() iterate
over these as well in a manner similar to the function above.  This
would be simpler and neater to use but perhaps not as general.

So what's your opinion on this?

Dimitris