If your __gc finlization metamethods makes more complex things requiring allocation of new resources, in my opinion the program has a design bug: these resources needed to free objects should have been allocated wit hthe object long before it gets dereferenced and then garbage collected for finalization. Finalization should not allocate memory except for very simple objects that have NO such __gc finalization routine (such as strings or simple preallocated buffers/indexed arrays).
Exactly. It seems to me the OP is concerned about temporary allocations during the __gc metamethod not being cleaned up until the next GC cycle. Well, welcome to the world of GC!