Re: [ Userdata ] __index aware of original lookup value

Dear Mr. Conner,

I'll do my best to communicate my understanding.

> "Not at all. Breaking down the call. Lua notices that myfoo is a
userdata. Since you are referencing a field, it looks to the metatable of
said userdata for the __index field. It's a function so it's called with
myfoo and "blah" as parameters. You do your magic and return a function.
Lua then calls that function with myfoo as a parameter. I don't see how you
can speed that up any."

There is a significant speedup if you replace __index with a table instead of a function. If you look at this link here ( http://satoren.github.io/lua_binding_benchmark/ ), there are some basic benchmarks of various frameworks and their call times for their implementations of C++ Member Functions ( I am the "sol2" cluster / row in the data ). You'll notice that we run fairly slow in terms of C++ member function call speed when compared to the fastest framework there. This puzzled me, as the process you describe for the function seemed like the only way to do it, so I looked into how other frameworks did it. After studying, I made a few changes and re-ran the benchmarks with the 2 different styles of __index for accessing member functions when you do `myfoo:blah()`: http://i.imgur.com/kCi2HoL.png ( the associated commit for the 'sol2 - table __index' row you see there is here[1] ).

The process you described above is the old method we used (the bottom half of that CSV of data) and it is markedly slower. I do not know why it is markedly slower. But it is, and it's observable, and that's where my original question came from.

While faster, the problem is the table implementation is inflexible: if I have a chained __index metamethod, as I explained in my first post, I end up with `myfoo.__index` and `blah` as the two parameters. `myfoo.__index` just comes to me as what I set it as: a table with a bunch of functions in it. I can't get the userdata value from it. This is where the table implementation falls apart, because if I want to access `myfoo.some_variable`, I will get `myfoo.__index` (a table) and "some_variable" (the key) and have no way to get at the userdata.

> "Could you please specify a sample of what you mean by this? I just want
to make sure I understand what you are asking."

A theoretical way for me to solve my problem would be to have the __index metamethod be a function (and incur the function call overhead), but to also somehow set fields on the userdata. E.g.,

int myfoo_new (lua_State* L) {

void* ptr = lua_newuserdata( /* sizeof etc. etc. */ );

/* call constructor / etc. */

// Set functions directly into the userdata itself, without having to rely on a metatable's metamethods

// this doesn't work, crashes the VM, can't set fields/functions on userdata

luaL_setfuncs( L, myfoo_functions, 0 );

// then, use the metatable exclusively for variable lookup (and __gc) - pay for the cost of the function call overhead as demonstrated by benchmarks
// only in the case of doing variable lookup

luaL_newmetatable(L, foo_metatable_name);

lua_pushcfunction(L, foo_variable_lookup);

lua_setfield(L, -2, "__index");

// set metatable to foo

lua_setmetatable(L, -2);

}

Typechecking and return checking omitted for brevity. Does this make clear what I'm trying to do / achieve?

[1] https://github.com/ThePhD/sol2/commit/0ee92c0142cbac162de4a29cc9340c89b3f64c23

On Wed, Feb 17, 2016 at 3:49 AM, Sean Conner <sean@conman.org> wrote:

It was thus said that the Great JeanHeyd Meneide once stated:
> Dear Mr. Conner,
>
> If I'm understanding how lua_get/setuservalue works, I need the
> userdata to make it work. Per your example, I would do it so the __index of
> the userdata's metatable is a function. Then I have access to the userdata
> and can use `lua_getuservalue` on the userdata to get the lookup table.
> This is more or less how I was doing it before (except instead of using a
> lua table with lua_getuservalue, I was looking up into a
> std::map/boost::map of functions/variables I had stored with the userdata).

Okay, I've done similar where manually looked up the value based upon the
key [1].

> The performance bottleneck wasn't the lookup (I had reduced the
> performance impact to nothing by switching from a C++ unordered_map to a
> map since there were a very small number of elements and then using
> "transparent lookup" introduced in the new C++ standard for `map` to not
> have to create a temporary string). The performance bottleneck for the
> member function implementation was having to return the retrieved function
> to lua so it could then call it again when doing `myfoo:blah()`. (If that
> makes sense).

Not at all. Breaking down the call. Lua notices that myfoo is a
userdata. Since you are referencing a field, it looks to the metatable of
said userdata for the __index field. It's a function so it's called with
myfoo and "blah" as parameters. You do your magic and return a function.
Lua then calls that function with myfoo as a parameter. I don't see how you
can speed that up any.

> Since I already had it working before, my goal here isn't whether or
> not I can do it (I know a way to do it, like the way you said, which makes
> me somewhat happy that I'm on the right track here!), but I'm trying to get
> the utmost performance possible, so that this code has no overhead compared
> to regular C code.
^^^^^^^^^^^^^^

I'm puzzled by this phrase. You are coding in C++, not C (first off), and
second, you're using Lua---there's going to be overhead. Secondly, the
__index function doesn't know how you are going to use the resulting
value---I mean, what's the difference between:

x = myfoo.blah

and

myfoo:blah()

Both will result in this (internally in the Lua VM):

t1 = getmetatable(myfoo)
if not t1 then error() end
t2 = t1.__index
if not t2 then error() end
if type(t2) == 'function' then
t3 = t2(myfoo,"blah")
elseif type(t2) == 'table' then
t3 = t2.blah
else
error()
end

The only difference past this point is

x = t3

or

t3(myfoo)

> As a side question, is it possible to set fields to a userdata
> directly?

Could you please specify a sample of what you mean by this? I just want
to make sure I understand what you are asking.

-spc

[1] https://github.com/spc476/lua-conmanorg/blob/faf0780d01700a45e2af5d52f2c732fa45092c1e/src/process.c#L715