Setting a __metatable field on the userdata's metatable is sufficient
to block access to the metatable from Lua (provided the debug library
isn't accessible) and that should be sufficient. You can set the
__metatable to false, or to the actual method table, or whatever.
The only time you'd need to filter in __index is if you've made the
metatable into the method table by setting it to be its own
__index. That's probably not a good idea for userdata; it's
almost certainly better to have separate method tables and
metatables. If you do that, you can skip the relatively expensive
type-check in unary metamethods (that is, everything but the
binary operators); you still have to do the check with methods.
While it's tempting to expose the method table as the value of
the __metatable key (as mentioned above), it's probably not a
good idea in practice. To start with, it violates the semantic
model: the method table is not the metatable, and a caller
looking for the metatable might expect to get the metatable
(for example, to access pseudometamethods like __tostring,
or __pairs if you use my __pairs patch.) A simple solution
is to duplicate such metamethods in the method table, adding
typechecks as necessary.
Second, if you're exposing the datatype to untrusted sandboxed
code, the shared method table provides a path for communication
(or havoc) between sandboxes.
Hiding the method table has disadvantages, though. It makes it
difficult for client code to extend the datatype with additional
methods written in Lua, which could be useful. (Consider the
case of augmenting the string library, for example.) One
possible solution, assuming that each sandbox has its own
luathread, is to put sandboxed method tables in the
thread-local environment (that is, the environment table
of the luathread) keyed by the base method table. The
datatype ('class' if you prefer) metatable's __index
metamethod is created with the base method table as
its environment; it can then use base table to lookup
the sandboxed method table and then look the key up
in that table; if it fails to find it there, it simply
looks it up in the base table. This slows down method
calls somewhat, but it's still reasonably fast.
