Just saying thanks for everyones help again. Tim's overview was very helpful and I have been experimenting with some changes to my code loosely based on Seans excellent code sample. This thread will be a goldmine for folks trying to solve this type of problem in the future. In fact someone else already chipped in and said it had helped them too. Having said that, it would be even better if this was documented as an example in the Lua literature somewhere. I think its a pretty common usage pattern in lots of applications.
I don't feel quite so bad about struggling with it now, as it turns out to be a fairly complex problem. In order to understand the solution you do need to understand a lot of Lua concepts. The C API, Tables, metatables, full userdata, lightuserdata, the registry, weak, strong keyed tables, creating userdata envinroments etc etc.
I don't claim to fully understand it in depth yet, but I have got it pretty close to working now, I can successfully call the "action" handler on my btn1 instance and pass the userdata self reference, and some arbitrary arguments. !!!! Yayy !!!!
The bit I haven't tackled yet is solve the gc issue, all 3 of you anticipated it as a problem. Because I am storing the userdata reference now in the registry, the strong keys stop the garbage collector from being called. I have debugged my code and can confirm that is what I am seeing. So I need to change my code a little bit to use my own weak keyed table. I am hoping I can figure that one out when I get around to it.
So I am getting there slowly, going to put an icepack on my brain now.
Tims reply: -->
Problem: You wish to invoke a Lua handler function when some external event occurs (such as a button click) that is associated with (in an abstract sense) an external “object” (might be a real object, or a window handle or similar). There may be many such objects, and you need the Lua code to be aware of “which” object it is dealing with by using basic OO techniques (that is, passing the Lua function some context).
Looking at your code and the thread, your basic idea is to use a full userdata, and register the various Lua callback functions using a metatable attached to that userdata item. You can then place state information (such as a window handle or a pointer to a C++ object etc etc) into the userdata. This allows C functions called from Lua to get access to the state (for example, to manipulate the window). So far, so good .. this is a common design pattern.
The problem, of course, is that when the event fires, your C/C++ code gets event information in the form of a window handle or some sort of object. The issue is then how to map this “foreign” reference back to the matching userdata. Without this, you cannot “find” the metatable, and cannot access the method, nor push the correct context arguments onto the stack before calling the method.
Solution: The solution is to create a map from the window handle (or whatever it is) back to the userdata, and the easiest way to do this is with a Lua table (you could do it with an external C++ collection class, but that has other problems). The Lua table can use the window handle as the table key, and have the userdata as the value, so a quick lookup (lua_gettable) will map your window handle back to the userdata. From there, you can access the metatable, and voila! .. you can call a Lua method passing it the userdata etc.
But where to put this table? The correct place is the Lua Registry, under a well-known key (use a string or a light userdata with a well-known pointer address). So the registry has one table (created by you), with keys that correspond to the native window handles (or whatever) and values that reference the full userdata.
1. Create (once only) the lookup table in the Lua registry, using some fixed key.
2. Create a metatable for the buttons, and store that in the Lua registry too (there are helper functions for this very purpose in the Lua API).
1. Do whatever is necessary to create the native button, getting (say) a window handle to the button.
2. Create a userdata in Lua (on the stack), populate it as necessary, typically with at least the window handle and any other state data you need.
3. Set the metatable you stored in the registry as the metatable for the userdata.
4. Save the metatable in your lookup table (the one in the Registry), using the window handle as the key and the userdata as the value.
1. When your C “click” code is called (or whatever) it will get a window handle (or whatever pointer to state you used as the key in step 4 above).
2. Using the window handle, lookup the userdata value in the lookup table in the Lua Registry.
3. Get the metatable for the userdata.
4. Lookup whatever Lua function you want to call (lots of different design patterns here).
5. Call the Lua function, passing in whatever data you want, presumably at least the userdata as “self”.
You will find helper functions in Lua to assist in storing metatables in the registry using string keys, and LuaL_callmeta() to call a metamethod etc. Pretty much everything else is just using the basic Lua table API.
One refinement here is to consider the lifetime of your userdata. Because you have stored a reference to the userdata in a Lua table, Lua will never garbage collect these items (because the table is holding a reference to them). This might be what you want; it means that Lua can never “lose” a button, since their lifetime is managed outside of Lua (Lua code cannot access the Registry and hence cannot delete items from your hidden lookup table). The above code thus assumes some kind of ButtonDelete() function that does cleanup, which should include code to remove the userdata reference from the lookup table (which will then make it a candidate for GC).
However, you may want the button lifetime to be controlled by the lifetime of the userdata (and hence by Lua code). In this case, you DONT want the lookup table keeping the userdata items alive. The solution of course is to set the lookup table to have weak values; this allows Lua to GC the userdata even when they are referenced in the lookup table (and, incidentally, scavenge the table entries when it does so, which is nice). In this case you probably also want to hook the __gc metamethod so that your code is called when Lua does cleanup the userdata to make sure any native button resources (window handles) etc are properly disposed of.
In terms of your example code, the buttonEventToLua() is where you will do all the registry table lookup, and this will give you access to the Lua button action function you need (and the state to pass into it when you call it).
Hope this helps!