Re: Extending C++ classes under Lua

[Root <listas@...>]

Hi.

On Fri, Sep 15, 2006 at 09:51:37PM -0700, Don Hopkins wrote:
> mark gossage wrote:
> > Hi folks,
> > We all seem to be looking at the same kind of thing. Basically extending a C++ object within Lua, and being able to call it within either C++ or Lua.  I think with a bit of effort, putting our heads together we should be able to crack this one.
> >
> > Here is a quick summary of the current state of SWIG-LUA:
> > When SWIG wrappers a class as a full userdata. The userdata holds a pointer to the object, a pointer to the SWIG_TYPE (an internal type structure) and a flag to point out if this object should be GC'ed.
> > It then adds a metadata to the object, which holds all the methods, as well as a bunch of functions to read/write the attributes (which is why you can access the attributes naturally). This metatable is held in the registry and actually shared between all instances of the object.
> >   
> Thanks for describing how SWIG-Lua works. It's a great piece of work, 
> and I'm glad to be using it.
[...]
> One cool thing about SWIG is that a lot of it is written with typemap 
> libraries. You can hook into typemaps and and extend them, and it's 
> flexible enough that the back-end can define new kinds of typemaps that 
> let the user hook into the wrapper generation in language specific ways. 
> It would be great if the SWIG-Lua back-end would generate Lua wrapper 
> files, so you could write typemaps to tailor the wrappers for whatever 
> kind of Lua object system you're using. Of course it should included a 
> library of typemaps to support the best practices (whatever those are -- 
> is there any consensus?). 

Right now on tolua++ the only requirement for peer tables (which are in
effect the lua part of the object) is that they be tables that can be
indexed with lua_gettable and lua_settable.. so basically the standard
'lua object syntax', instance:method(), which is the most common type of
object.
For calling virtual methods (I'm not sure if youre' planning for that
too), it only has to be indexable (it's usually the userdata directly,
since it's passed as the 'self' argument to the lua method)

[...]
> I haven't been able to figure out by reading the code if or how lua++ 
> makes sure there are never two userdata's referring to the same C++ 
> object. I think it's worth the extra effort it takes to ensure that
> there's a 1:1 mapping between C++ objects and wrappers (interning
> userdata), because that enables you to hang extra properties off of the 
> wrapper objects, and they won't go away or get confused with multiple 
> wrappers around the same object.

We used to have a global table, 'tolua_ubox' in the registry, with a
lightuserdata with the pointer of the object as key, and weak values. The
problem with using only one table is that 2 objects can have the same
address and different types, for example:

struct A {
	struct B {
		int n;
	} b;
} a;

&a and &a.b (and &a.b.n) whill have the same memory address.. To solve
this we put a tolua_ubox on the metatable of each base class (a class
which does not inherit from anything), and put a reference to it on every
class that inherits from them (so anything that inherits from 'Widget'
will use the same ubox).

You also have to keep track of the type of the userdatas you store there..  
If you push the same object twice, first with type 'Button' and then with
type 'Widget', you have to figure out which type is the most
'comprehensive' for the object, since you might have other references to
it inside the luastate.
This is mostly done on tolua_pushusertype

Ariel.

> > I have commited in the the SWIG CVS a new file which helps for writing callbacks:
> > http://swig.cvs.sourceforge.net/swig/SWIG/Lib/lua/lua_fnptr.i?revision=1.1&view=markup
> > (there is also an example of using it in CVS)
> >
> > This might be of you for some of you. 
> >
> > Can some of you help me with some more idea's on how to address the rest of the issues?
> >
> > Thanks,
> > Mark
> >   
> Thanks for posting the typemaps -- they're so cool! Your SWIGLUA_REF 
> typemap is similar to the one I wrote to handle references. But I just 
> assumed there was one global interpreter named "L", and stored plain 
> integer Lua object reference ids in my C++ members (making a "typedef 
> int LuaRef" to let SWIG know what I meant). And I had to call the 
> equivalent to your swiglua_ref_clear function in my C++ object's 
> destructor to make sure the corresponding ref's get cleaned up.
> 
> The primary LuaRef I was using for each C++ object was a reference to 
> itself in Lua-land (the peer object), so it was easy for the C++ object 
> to pass the peer object back to the handlers as the "self" argument, and 
> to access Lua properties attached to the peer object (like instance 
> variables and callback methods), and stuff like that.
> 
> An alternative more gc-friendly approach would be to use a weak table 
> like tolua++ is doing. My stupid objects are owned by the application, 
> created by a factory and destroyed through helper functions, and I don't 
> currently intend for the Lua programmer to create and destroy them 
> directly with new and delete (at least at this stage) or for the Lua GC 
> to collect them. Of course that would be nice once we figure out the 
> best way to do that, but it would be best to support both approaches to 
> object ownership (the application owns, creates and destroys the 
> objects, versus the Lua programmer can call the class's new and object's 
> delete methods, and the Lua GC controls their lifetimes.
> 
> I'm still learning Lua and trying to get my head around it, especially 
> how the meta-object programming stuff works, and how metatables, 
> userdata and native code extensions interact.
> 
> One question I have about Lua OOP in general is: why (in the examples on 
> the Wiki) is there both a metatable, and also a separate method table 
> (which the metatable points to with its __index attribute)? This seems 
> kind of wastefully JavaScripty, and not as minimally Selfish as I'd 
> expect. Is there any reason not to put methods and class variables 
> directly into the metatable, and dispense with the extra method table in 
> the metatable's __index attribute?
> 
> Lua is a lot like Self. But one difference is that Self lets you mark 
> any slot as inheritable, so you can implement multiple inheritance by 
> having multiple parent slots (with different names of course). Like the 
> userdata has both a metatable and an env slot (but the env slot is 
> general purpose and not automatically inherited from, so the metatable 
> has to know how to delegate to the env for "dual inheritence" to work). 
> What we're trying to do by binding together Lua "peer" object tables 
> (the Lua object) with Lua userdata objects (the C++ object), C++ 
> metatables (the C++ behavior) and Lua scripted classes (the Lua 
> behavior), seems a lot like multiple inheritance, where we first check 
> the "peer" table for user defined attributes/methods, then check the Lua 
> class for scripted attributes/methods, then the C++ metatable for native 
> code attributes/methods.
> 
> userdata:
>     object => pointer to C++ object, represents object binary data and 
> native code
>     metatable => C++ metatable behavior, dispatches attributes and 
> methods into binary data and native code
>     env => Lua peer table, represents scripted side of object
>        peer's properties: scripter defined instance variables, 
> callbacks, etc.
>        peer's metatable => Lua class behavior
>           class's properties: scripter defined class variables and methods
>           class's metatable => Lua superclass behavior, etc...
> 
> global weak value dictionary:
>   lua peer table (strong) => userdata (weak)
> 
> Then the C++ object needs a way to get to the corresponding userdata. 
> That could be:
> 
> global weak value dictionary:
>   integer address of C++ object => userdata (weak)
> 
> Or it we could use the (more efficient?) approach of putting an integer 
> reference to the userdata in the C++ object, like I was doing with 
> LuaRef's. But that may leak memory (and it imposes on the design of the 
> C++ object, making it harder to wrap uncooperative code, i.e. libraries 
> written by other people), so it might be better to use a weak dictionary 
> to map from C++ object addresses to userdata's. (Or does Lua intern 
> userdata's automatically? That'd be nice!)
> 
>     -Don