The example of the problem comes from a design decision I made when dealing with Userdata, particular pointers to structs. Consider the following:
struct thing {};
thing v; thing* array_of_things[3] = {&v, NULL, &v};
Then, you serialize this into a table in Lua. The typical implementation would be to simply iterate through the array, and then push the value of the item at `array_of_things[i]` into Lua, as a userdata, and put that into a table. this makes the following happen:
print(
c_api_created_table[1] ,
c_api_created_table[2] ,
c_api_created_table[3]
)
----
0BE57DAD, 000000000,
0BE57DAD, or something
However, users who do this would also expect the following to work decently well:
if c_api_created_table[2] then
print('totally here') -- prints 'totally here', but that's not what the user expects!
else
print('totally NOT here')
end
The problem is that pushing a NULL pointer as a userdata value doesn't play nice with `nil` semantics and truthiness in Lua. `c_api_created_table[2] == nil` also would not play ball very nicely: it results in false, which sends everyone on a wild ride. It is also impossible to override the equals operator for such a thing, due to Lua's metatable rules for __eq [1]. You have 2 choices: either create a Sentinel Null proxy value, OR you push nil when you detect NULL pointers.
In order to play nice with Lua semantics and Prior Art[2] in these cases, detecting C or C++ NULL and appropriately doing a `lua_pushnil` is how many libraries, wrappers and frameworks have behaved for a long while (though some others did choose a sentinel USERDATA_NIL approach as well).
Doing that, however resulted in ANOTHER problem: tables under pre-Lua 5.4work1 with nils would essentially cut a sequence "short" [3]. This made it impossible to use regular tables with Userdata Pointers But With Nil In the Mix. ipairs and all those class of functions -- and many wrappers that took Lua's definition of sequences to heart and emulated its behavior -- all would fail[4].
Under the new paradigm of allowing "nil", everything Just Works™ now. The user gets their expectation that `nil` gets pushed for NULL userdata pointers, and sequences no longer have to be broken because of taking this design decision. It's a really good win-win.
[1] - Emphasis my own: "__eq: the equal (==) operation. Behavior similar to the addition operation, except that Lua will try a metamethod only when the values being compared are either both tables or both full userdata and they are not primitively equal. The result of the call is always converted to a boolean."
[2] - https://github.com/ThePhD/sol2/issues/289
[3] - https://github.com/ThePhD/sol2/issues/383 [4] - This reminds me that I have to add some custom handling to my table serialization routines. I was going to start checking if the type had a metatable, and if that metatable has a `__len` entry, to use that to get a size hint which will help for proper iteration, even in the presence of `nil`. This would help patch away some of the issues of iterating through a custom container that may contain `nil`...
