Re: tables holding nil: another way to look at the question itself

[Jerome Vuarand <jerome.vuarand@...>]

2009/8/4 Cosmin Apreutesei <cosmin.apreutesei@gmail.com>:
> Suppose I have to wrap function f in function g. I don't care for f's
> arguments, just need to pass them along as g receives them. Can I just
> ignore the feature of arg. count in this case? My own function g
> surely don't need the feature, but the function f, written by someone
> else, might be sensible about arg. count, forcing me to account for
> this feature too if I am to implement g() correctly. That's what I
> meant when I said I have to carry the abstraction leak everywhere --
> after all, that's what makes it a leak.

I think your example doesn't show what you're trying to explain. If
you want g to forward all its arguments to f, you would write g like
this:

function g(...) f(...) end

Here the argument count is not visible, and it could as well be
unavailable, you wouldn't notice. So it's clearly not leaking. It's
not intrinsically part of ... (you could have ... without select('#',
...)).

And ... in g argument list is the only way to forward all arguments to
another function. Even if you need to pack the arguments in a table,
you don't need select('#', ...) since you could use next/pairs to get
the biggest non-nil argument index to pass as third parameter to
unpack.

> I see pairs() as an abstraction leak too, allowing me to "see" a small
> set of the invinite set of keys. Example: say I need an unordered list
> of objects, any of which can have a value attached. This can be
> represented in two obvious ways:
>
>  1) { {obj1, obj2, ... }, {[obj1] = obj1_value, [obj2] = obj2_value, ... }
>  2) { [obj1] = obj1_value, [obj2] = obj2_value, ... }

Your example 2 is clearly a misuse of table for your needs. You want
to represent a finite set of key-value pairs, while tables are
infinite. You need a convention to represent a finite set of pairs
with a table. Either store the list of keys in another table (like
your solution 1), or forbid a value to be used in key-value pairs of
the set, and assign that value to all the keys *not* in your set, as a
"not in the set" tag (and obviously using nil as the tag makes your
life much easier).

That handles nil values. For the nil key, you have to find another
mechanism, but since the number of pairs in that case is finite (only
1), it's relatively easy.

Also I assumed your set of pairs don't allow several pairs to have the
same key, so as to make them somewhat compatible to tables. If that's
not the case you have to do more complex stuff anyway, and the
infinite aspect of tables is less of a concern anyway.

> The first allows objN_value to be any value, including nil, and
> doesn't require pairs() to exist. The second representation is made
> available by pairs(), and that's the one I see most of the time in Lua
> code, usually with nil values "fixed" with a boolean false value. When
> I see code like this I understand why some people believe tables
> "can't hold nil" :) Without pairs(), you couldn't have representation
> #2, and so no issues with nils, but pairs() leaks hidden information
> about tables, specifically the list of non-nil keys, a table which
> itself can't hold the nil :) I'm not pleading ditch pairs() here, it's
> probably too useful, but it all falls down on poor nil.

pairs doesn't *leak* the list of non-nil keys, since there is no
reason that information should be hidden in the first place. If you
consider tables to be infinite, you cannot iterate over them in a
finite time. However if you know a subset of your table key-value
pairs is finite, it's useful to be able to iterate over them. It
happens that most of the time the set of pairs with a non-nil key is
finite, so pairs conveniently gives you access to these pairs.

And since iterators are not required to terminate, pairs is not even
leaking the fact that the current implementation of tables don't allow
one to have an infinite set of pairs with non-nil keys (some new
syntax or library would be need to create these anyway).

>> Also the read-only nil-key pair is a feature of tables, while index
>> and newindex let you build other types of objects (eventually with
>> different semantics) on top of tables. Keep in mind that index and
>> newindex in tables are exceptions in the real of metamethods, because
>> they let you override a behaviour that is otherwise perfectly defined
>> and valid.
>
> Metatables should allow me to override the implementation of a table,
> they shouldn't let me corrupt its semantics (whatever that is) or
> things will break.

I somewhat agree with the second part of your statement, index and
newindex let users corrupt the semantics of a table. IMHO they
shouldn't be called on tables, since they break the general rule that
metamethods are only called on undefined operations (ie. which would
throw an error). Instead, if users want to create new data types, they
should use full userdata. And ideally some new mechanism to create
pure-Lua (ie. without C) zero-sized userdata should be created
instead, based on the newproxy experiment, eventually as a separate
type from userdata (what about 'object'?).