Re: Userdata finalization order

[Francesco Abbate <francesco.bbt@...>]

2010/10/18 Roberto Ierusalimschy <roberto@inf.puc-rio.br>:
> There is a difference between "finalization" (calling the __gc
> metamethod) and "disposal" or "collecting" (actually freeing the object).
>
> Any object that is going to be finalized, plus any object accessible
> from it, is never collected before the finalization, no matter the
> finalization order. In particular, when Lua closes a state, it calls all
> finalizers before disposing any object. There is no way to crash Lua by
> accessing Lua objects during finalization (bar bugs in Lua).

Hi Roberto,

thank you for taking the time to discuss this subject.

Regarding you comment, the bug I was talking about is because actually
you *can* access userdata that have been finalized if they are in a
weak table as keys. As you know Lua will not remove immediately the
finalized object from the table if they are weak keys but it waits for
the next GC cycle. I will still think that this is a bug unless you
explain why it is not. Note also that this "behaviour" is absolutely
undocumented.

> C/C++ objects is a different matter. They may be (and often are)
> disposed by the finalizer of its proxy in Lua. So, if the finalizer for
> object A tries to access C/C++ data associated with object B, and the
> finalizer for B disposes that data, then you should ensure that B is
> finalized after A. No manipulation of references or weak tables will
> solve your problem here. You have basically two solutions:
>
> - If A cannot live without B, then it should be created after B
> (otherwise it would have to live without B until B is created).
> If it is created after B, Lua ensures that its finalizer will be
> called before B's (as pointed out by E.T.)
>
> - If A can live without B, then it could avoid accessing B's data during
> its finalization. For instance, B finalization could mark in B's Lua
> part that the object is already finalized, by changing its pointer to
> C/C++ data to NULL. The finalizer for A then checks whether the pointer
> is NULL before accessing that data.  (Remember that the pointer itself,
> stored in the userdata memory, is always accessible, even after B's
> finalization.)

Well, Roberto, here you are proposing some workaround to a Lua
problem. The first proposition is in many case not acceptable, in most
applications the object can be created in any order and the links
between then are established at run time. To impose always a specific
order of creation is simply too much restrictive or just impossible in
many applications.

Talking about the second proposition, you are saying that we should
avoid to access referenced objects during finalization because these
could have been already finalized. I'm sure you understand that this
principle is not part of the normal C++ programming practice. Normally
when an object is finalised it also releases all of its resources and
in some cases the resources can be other objects. The problem here is
"how do you know that the secondary object have already been finalized
?" There is no obvious way to know that. I believe that what you
propose is technically possible in many cases but it does impose some
extra constraints to the C++ design of the code and an additional
complexity of the problem.

I believe that in this area Lua could be *really* improved and the
demonstration is that many C++ programmer bypass the normal Lua memory
management by creating a shallow userdata (just a boxed pointer) and
use reference counting. This is obviously an extra mechanism over
Lua's GC that is needed to ensure correct memory management. I believe
that a *native* Lua mechanism to handle these cases could really
improve and ease the integration of C++ applications.

-- 
Francesco