lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


Oops, I goofed up some of the footnotes. The "all would fail[4]" should be "all would fail[3]", and the [4] should just be a P.S., aha.

On Sun, Mar 18, 2018 at 4:28 AM, ThePhD <jm3689@columbia.edu> wrote:
The example of the problem comes from a design decision I made when dealing with Userdata, particular pointers to structs. Consider the following:

     struct thing {};
     thing v;
     thing* array_of_things[3] = {&v, NULL, &v};

Then, you serialize this into a table in Lua. The typical implementation would be to simply iterate through the array, and then push the value of the item at `array_of_things[i]` into Lua, as a userdata, and put that into a table. this makes the following happen:

     print( c_api_created_table[1] , c_api_created_table[2] ,  c_api_created_table[3] )
     ---- 0BE57DAD, 000000000,  0BE57DAD, or something

However, users who do this would also expect the following to work decently well:

     if c_api_created_table[2] then
          print('totally here') -- prints 'totally here', but that's not what the user expects!
     else
          print('totally NOT here')
     end

The problem is that pushing a NULL pointer as a userdata value doesn't play nice with `nil` semantics and truthiness in Lua. `c_api_created_table[2] == nil`  also would not play ball very nicely: it results in false, which sends everyone on a wild ride. It is also impossible to override the equals operator for such a thing, due to Lua's metatable rules for __eq [1]. You have 2 choices: either create a Sentinel Null proxy value, OR you push nil when you detect NULL pointers.

In order to play nice with Lua semantics and Prior Art[2] in these cases, detecting C or C++ NULL and appropriately doing a `lua_pushnil` is how many libraries, wrappers and frameworks have behaved for a long while (though some others did choose a sentinel USERDATA_NIL approach as well).

Doing that, however resulted in ANOTHER problem: tables under pre-Lua 5.4work1 with nils would essentially cut a sequence "short" [3]. This made it impossible to use regular tables with Userdata Pointers But With Nil In the Mix. ipairs and all those class of functions -- and many wrappers that took Lua's definition of sequences to heart and emulated its behavior -- all would fail[4].

Under the new paradigm of allowing "nil", everything Just Works™ now. The user gets their expectation that `nil` gets pushed for NULL userdata pointers, and sequences no longer have to be broken because of taking this design decision. It's a really good win-win.

[1] -  Emphasis my own: "__eq: the equal (==) operation. Behavior similar to the addition operation, except that Lua will try a metamethod only when the values being compared are either both tables or both full userdata and they are not primitively equal. The result of the call is always converted to a boolean."
[2] - https://github.com/ThePhD/sol2/issues/289
[3] - https://github.com/ThePhD/sol2/issues/383
[4] - This reminds me that I have to add some custom handling to my table serialization routines. I was going to start checking if the type had a metatable, and if that metatable has a `__len` entry, to use that to get a size hint which will help for proper iteration, even in the presence of `nil`. This would help patch away some of the issues of iterating through a custom container that may contain `nil`...


On Sat, Mar 17, 2018 at 7:29 PM, Sean Conner <sean@conman.org> wrote:
It was thus said that the Great ThePhD once stated:
> Having nil in tables is also an immense conceptual boon for people who work
> with the C API and userdata. Having a userdata-specific `nil` was extremely
> sad for when you wanted to store your userdata in a table, because it
> messed with usual language semantics of `if t[key] then ... end`. Now, it
> works out of the box and both pairs() and ipairs() behaves just fine, and
> `nil` can be used as a proper `nullptr`-alike value!

  I'm not following, and I'm someone who uses the C API and userdata.  Can
you give an example of the problem?

  -spc