What is missing however is that when the new metatable replaces an existing metatable, the former metatable gets dereferenced, so its counter of references will be decremented, and the former metatable could become garbage collectable at any time after this call, or possibly during this call. As well the new table (attached to an object as a its metatable) gets one additional reference, and so it won't be garbage collected during the call or just after it, as long as the object remains accessible to the Lua VM.
But there's a possible race condition if you happen to overwrite an existing metatable by itself: detaching the existing table could make it garbage collectable in the middle: the engine should make sure that the decrementation will not occur on the detached table and the incrementation will not occur on the attached table if they have identical pointers: setting a metatable to the current metatable should have no effect at all.
The garbage collector does not do reference counts. It uses one of the many variants based on Dijkstra's tricolor scheme for incremental tracing garbage collection. It traces all the objects starting from the Lua VM registers and the stack used by C code interfacing with Lua. When you call 'lua_setmetatable' there's a pointer to the metatable on top of the stack, so it won't be freed. The garbage collector in Lua does some bounded amount of work each time storage gets allocated, so the work is interleaved with the operation of the program. One advantage of a tracing garbage collector is that it does not get affected by circular references. Objects that are not referenced anymore get collected eventually, but if the program does not allocate much storage this can take a while. This is the reason the finalizer (__gc metamethod) does not get called immediately, but only when the garbage collector has done enough work to prove the object is not being referenced by any strong reference. It's also the garbage collector's job to set weak references to nil once all the strong references are gone.
Here's the CACM paper:
Edsger W. Dijkstra Leslie Lamport A. J. Martin C. S. Scholten E. F. M. Steffens
Communications of the ACM 21 | November 1978, Vol 11: pp. 966-975
I'm sure there are major differences between this scheme and Lua's, but it'll get you started 😊
Gé
(In case anyone wonders how reference counting should be implemented correctly: FIRST you increment the reference count of the new value, and THEN you decrement the reference count of the old one. If they're the same object the reference count goes up one and then goes down one, it never goes to 0, so no collection occurs, no harm, no foul. This is how C++'s std::shared_ptr<T> works, ignoring weak pointers and niggling details such as thread safety. There is no race if you order the operations correctly.)
