This equates to a call to luaV_settable, with the table pointer referring to an entry on the stack, and the key being 'x'.
I believe that Peng's suggestion is a good one to consider, regardless of whether or not your understanding match.
If you say "my_vec = vecA + verb", you can overload the "__add" operator to return a new vector object.
In the case of "myvec.x = 11", you could make that an error and instead require something like "myvec = myvec:set_x(11)".
Also, I've overloaded the type library to do custom type checking. It's not the best interface in the world, but it does allow me to do stuff like "typex(myvec, "vector")" or "typex.tostring(myvec) == 'vector'".
I don't pretend to know your constraints, but for myself, every time I've benchmarked it, Lua's "normal" methods for doing things, even with tables and metamethods, have been fast enough. That is to say, it is worth benchmarking (not just micro benchmarking), if you have not done so already.
All of this is short of the utopia of first-class types. But like i said, it may be worth considering as an alternative to modifying the VM.
