Re: Userdata finalization order

[Francesco Abbate <francesco.bbt@...>]

2010/10/17 Petri Häkkinen <petrih3@gmail.com>:
> Yep, that sounds sensible. It's just that tracking all those dependencies is
> a lot of work (there are dozens of types/classes) and the binding is already
> several thousand lines of C++ code so I was hoping for an easier solution.

Hi,

I've got exactly this kind of problem with Lua and C++. Lua is not
able to ensure a finalisation order of the userdata objects. The only
solutions I know of are:
- you use light userdata objects and you leave the memory management
to C++ by using
  reference counting => almost everyone is doing like that
- you use weak tables to mirror the relations between C++ objects so
that Lua GC can
  respect references and finalisation orders

Please note that using an environment table for userdata does not
work, I will explain why. Suppose that an object "a" depends on a
second object "b". In order to ensure the finalisation order you set a
reference to "b" in the env table of "a". This seems to work but
suppose now that both "a" and "b" becomes unreachable. What happens is
that Lua GC will free both of them regardless of the environment table
of "a" and so the finalisation order can be violated.

Please note that weak tables also have a major problem because when
the program terminate Lua will do something very coarse with
userdatas: it will free all the userdata all together regardless of
anything else. It is evident that this will screw up the finalisation
order that you was struggling to ensure with the weak tables.

The only solution that I've found to this problem was an ugly hack
made just before Lua close. The other solution would be to call "exit"
before Lua has terminated but this is equivalent to admit your defeat
:-)

For me it is evident that this is an area where Lua should be
improved. We are talking about *real* problems and not syntax sugar or
more convenient syntax or anything fancy like that.

Some times ago I've proposed a very simple modification to Lua to fix
this problem: add a function lua_addref to add a reference from a
userdata to another userdata. The idea is that every userdata will
keep a list (possibly empty) of others userdata that it depends on.
Then the GC will ensure the proper finalisation order just by
inspecting for every userdata the list of references. Unfortunately
this suggestion was dropped without too much discussions, people told
me that env table for userdata where just enough.

Unfortunately Lua authors doesn't seems to be interested to fix this
problem. As far as I understood all the people that are working with
complex C++ code use light userdata and C++ reference counting.

I have also reported a bug related to some userdata on weak tables as
keys that are finalised but still kept in the weak table. See this
thread:

http://lua-users.org/lists/lua-l/2010-08/msg00605.html

it was clear that it is a bug and Mark Hamburg helped me to fix the
problem easily but the problem was not fixed in Lua neither Lua
authors made any comment about it.

-- 
Francesco