__pairs isn't what we need

[Quae Quack <quae@...>]

I've had this in the back of my head for ages now, and need to get this out before __pairs becomes official.

A __pairs isn't what we need:
pairs has never existed on the C api side (and it should stay this way): you use lua_next.
==> Even with the new __pairs metamethod, true proxy tables can not be created, as they won't be respected by C functions.

What we need is a __next metamethod:
==> obviously next() will respect it
==> pairs should return returns a next function that respects the __next metamethod.
==> lua_next will respect the __next metamethod (so C code will be able to iterate over our proxy tables/userdata/etc)

Concerns:
__next would require a metamethod lookup for each iteration:
==> This could be a bit of a performance problem, my proposed solution is having pairs check for a __next metamethod on it's argument, and in it's absense, return a lua_rawnext
Should we provide a lua_rawnext and rawnext accessible from lua?? (following the tradition set by rawset,rawget and rawlen)

Example:
using __next for a proxy table:
---------------------------------------
mytable = { "foo" , "bar" , "baz" }
proxytable = setmetatable ( { } , {
  function __next ( o , lastret )
    return next ( mytable , lastret ) -- Hooray for tailcalls
  end
} )
---------------------------------------

Notes:
This cannot be implemented/replaced in pure lua: C functions need to be able to iterate over our objects!

Other rambling:
ipairs has a similar issue at face value (with it's hidden "inext" iterator); but I propose we make it respect __len.
alternativly, I argue for the removal of ipairs alltogether: it's functionality can be replicaed with a number for - and you can always make your own ipairs function:

---------------------------------------
for i=1,#o do
  local v = o [ i ];
  <code>
end

function ipairs ( o )
  return function ( o , i )
    i = i + 1
    local v = o [ i ] -- use a rawget??
    if not v then return nil end
    return i , v
  end
end
---------------------------------------
^^ This ipairs uses the old behaviour of iterating until the first nil in the table, it also provides the hidden "inext" function.

At the very least, moving ipairs to table.* would make alot of sense....

Regards,
Daurnimator