Callbacks, upvalues and memory management

[Gregory Bonik <gregory@...>]

Hello, 

I am writing Lua wrappers for my set of C++ classes. All classes are
allocated in the heap and have an internal reference count. Each time
when a pointer to C++ object is pushed to Lua stack (as a full
userdata), its reference count is incremented. In the __gc metamethod,
the reference count is decremented. Such strategy looked valid until I
started to implement callbacks.

Suppose that a C++ class Button has a method called on_click which sets
a function to be called on some event, say, mouse click. For sake of
brevity, let's suppose that it accepts a reference to a Lua function
created by luaL_ref:

void on_click(int function_ref);

A wrapper for this function could be written like this (checks
intentionally omitted):

int wrap_Button_on_click(lua_State* L) {
    Button* self = (Button*)lua_touserdata(L, 1);
    lua_getfield(L, LUA_REGISTRYINDEX, "refs");
       -- "refs" is table with both strong keys and values
    lua_pushvalue(L, 2); -- push function to the top
    int ref = luaL_ref(L, -2); -- add a ref to the function
    self->on_click(ref);
    return 0;
}

Now consider a fragment of Lua code:

local b = Button()  --create new button
b:on_click(function() b:some_method() end)
b = nil

The problem is that an anonymous function passed to on_click() has an
upvalue which refers to the button, which implies that the button's __gc
method will not be called while the function exists. But the function is
referenced in the "refs" table and will not vanish until the destruction
of C++ object. Hence we get a circular reference which causes a memory
leak.

Is there a solution to this problem?

Making values of "refs" table weak is not an acceptable solution, since
an object can be created and used inside C++ code, e.g. like this:

local b = get_button() --get a button already created and used by C++
b:on_click(function() b:some_method() end)
b = nil

Thanks in advance.

-- Gregory Bonik