Re: OOLua version 1.2.2 released

[Daniel Wallin <daniel@...>]

liam mail wrote:
> 2010/1/12 Daniel Wallin <daniel@boostpro.com>
> Daniel thank for the comments yet some of your claims have flaws.
> 
>> liam mail wrote:
>> [...]
> 
>> Since you are making such a fuss over this I'm going to comment on your
>> benchmark.
>>
> Well OOLua was designed with speed as a consideration. I did not realise
> that a single link was making a fuss yet showing off the it is quicker than
> other libraries tested.

This is hardly the first time you post links to the benchmark results.
It bugs me because your results makes the other binding libraries come
off as really slow, when in fact the only part that is faster in OOLua
is the trivial table lookup.

It's going to be difficult for someone reading your numbers to
understand what the real differences are.

>> First, it isn't very clear what you are trying to measure. For instance,
>> the first benchmark - "access" - is simply calling two member functions.
>> Why is it called "access"?
> 
> 
> The first speed tests where against the hand written code from Game
> Programming Gems 6, if you look the code used and the test name is
> surprisingly similar.
[...]

I don't really care where the code came from, I'm saying it's named badly.

>> But more importantly, *all* of your benchmarks are mixing in the cost of
>> fetching a member from an object. This is the primary reason your code
>> appears to perform better.
> 
> The reason is that normal usage (from my experience) is not to cache the
> function as it is an object orientated binding and given the fact that OOLua
> was designed not to have runtime overhead of function lookup (which I
> comment about in the following link and at other locations
> http://code.google.com/p/oolua/issues/detail?id=4&can=1)I feel it is correct
> to test it like so.

If you all the same member function repeatedly and it shows up on the
profile, you cache the lookup. This is true in raw Lua, when using OOLua
and when using any other binding library.

My point is that the results are misleading because they don't mention
this. You could very easily have another benchmark that measures the
time it takes to lookup a member.

> I could add the test you suggest, yet I am actually
> waiting for yourself to make the tests you have claimed in the past that
> luabind 0.8 was on par with swig. To my knowledge you have not made these
> public yet.

No, I haven't. I also haven't published any numbers. I mentioned luabind
0.9 in general being close to Swig in speed because someone posted a
benchmark on the luabind mailing list comparing some especially weak
areas of luabind.

I don't have time to make my suite work on anything but my machine. It
tests a few more binding libraries, and it generates the binding code
for some of them in the build step so there's some work to make it
usable outside my environment.

Either way, it doesn't really matter, because your code shows the same
results.

>> The reason lookups in OOLua is so fast is that __index doesn't call into
>> C++. Both Swig and luabind does, because they both support "properties".
>>
> I have never claimed that OOLua provides the same functionality as other
> binding libraries and have suggested that when someone is asking for a
> specifc piece of functionality that Luabind maybe a better alternative.
> After saying that I am a little unsure what you define as a property? Do you
> mean C++ side public member variables or Lua side?

"properties" are functions that are invoked when accessing something
that looks like a data member:

  x = X()
  x.value = 1 -- calls a set-function
  y = x.value -- calls a get-function

Supporting this is the reason both luabind and Swig (and tolua, ..)
roundtrips into C/C++ before making the real table lookup.

>> Further, there is essentially no error checking. For primitive types,
>> the error check consists of an assert(), so in the release build there
>> is no checking at all. For both Swig and luabind, calling a function
>> with the wrong parameter type will give a meaningful error, and at least
>> for primitive types there is no way of turning this off. For luabind at
>> least, this is by design.
>>
> This is true. OOLua assumes that you are passing the correct primitive
> types to functions which you call and there is only a check to see if the
> data is of the types which Lua provides using lua_isnumber et al in debug
> mode. Yet how do you check if an input is a char or int?

You don't, because they are represented by the same type on the Lua
side. Isn't that obvious?

>> OOLua also doesn't verify that the userdata it gets from Lua is actually
>> a OOLua-wrapper. But to be fair, neither does Swig. I guess not doing
>> this might be OK if you can completely trust the Lua code that you run.
>> luabind 0.9 has no way of turning this check off, but my working copy
>> does (it's a one line change).
>>
> Actually it does. It first checks to see if the current type is the type
> requested and then travels the *hierarchy* to check if any of the base
> classes match the requested type.

You missed my point. Consider:

  x = create_arbitrary_userdata()
  instance.f(x)

luabind verifies that the userdata passed in is actually created by
luabind. OOLua and Swig does not.

-- 
Daniel Wallin
BoostPro Computing
http://www.boostpro.com