Hi,
If you are working on a self-contained project, you can probably obtain most of the functionality that you want by adding a metatable for light userdata that can override __eq, then you can use these to hold your "integers". This would allow you to work with "unboxed" small integers (since "void *" is a uintptr_t) if that's a performance issue. As an added bonus, this fixes Andrew's concern: the light-userdata metatable __mul and __add can detect overflow and substitute a full userdata MPI object.
Alternatively, you could just patch it so that all integer arithmetic is bignum by default (I think Python does this?). This would probably be the fastest approach, and allows you to optimize more aggressively for the situation you're concerned with. Lua is meant to be modifiable.
However, if you're distributing this, I think that people are correctly very wary of making the "==" operator do something other than checking for equality. It can introduce security issues in old code, for example, if I pass an object with a metamethod that overrides "==" to a function that uses this to validate input. One alternative would be to introduce a new "equivalence" operator which can be overridden -- _javascript_ and PHP both did this, IIRC. An easy hack would be to just use __bxor (~), which already looks kind of like equality.
-Mason