Re: 'table' as fallback for tables

[Andrew Starks <andrew@...>]

On Tue, Jun 28, 2016 at 8:39 AM, Adrián Pérez de Castro
<aperez@igalia.com> wrote:
> Hi there,
>
> Just sharing my opinion below...
>
> Quoting Tim Hill (2016-06-28 08:44:12)
>
>> > On Jun 27, 2016, at 9:15 PM, Philipp Janda <siffiejoe@gmx.net> wrote:
>> >
>> > Am 28.06.2016 um 05:29 schröbte Tim Hill:
>> >
>> >> so adding 8 bytes per table (for the length) isn’t much, and might
>> >> even be free since most heap allocators round allocations up to
>> >> multiples of 16 bytes or more these days anyway.
>> >>
>> >> My personal feeling is that we need a first-class explicitly set
>> >> length. If this isn’t set for a given table (is nil), then Lua uses
>> >> current # behavior, as do things like table.pack().
>> >
>> > Mixing `.n` *and* sequence-style `#` will just combine the confusion, ugly corner cases, and undefined behaviors. Functions that take `.n`-style tables should just throw an error if there is no valid `n`.
>>
>> I agree, and I would prefer to get rid of “.n” asap, but at the same time how much existing code will break?
>
> Personally, I would also like the “#t” vs “t.n” confusion gone, as it is
> probably Lua's ugliest wart. For a decade I have been avoiding having to use
> “.n” in all my code whenever possible, to avoid shooting myself on the foot.
> This sometimes has lead to reworking my data structures in ways that arrays
> with “nil” values are not needed, or their length is not relevant or needed
> for calculations — but that's not always possible.
>
> At this point I do not really mind which solution is adopted, as long as the
> matter gets solved. I'll be happier, even when having to rewrite some bits of
> code ;-)
>
> Best regards,
>
> --
>  ☛ Adrián “2¢” Pérez

Because `#t` and `__len` exist, there is a perception that Lua "has
sequences". To me, in order for that to be true, there would need to
be something that enforced the limitations of a sequence. Instead,
it's more accurate to say that "Lua's tables can contain a sequence
and there are some nice notations to help you find their length."

Varargs are not strictly sequences because they can contain nils and
so `table.pack` should not be said to return a sequence with `n` as
the length specifier. `n` is the number of arguments.

The case could be made that `table.pack` should use a shared metatable:

{
    __len = function(self)
        return self.n
    end,
    __newindex = function(self, i, v)
        if not self.n or ( i > self.n ) then
            self.n = i --how does one shrink the array?
        end
        rawset(self, i, v)
   end,
  __ipairs = function(self)
      --return ipairs iterator using n
    end,
}

Here we still have `n` because you need some field in the table to
distinguish length. If the desire to purge the table of `n` field,
then you need a unique metatable for each value returned by
`table.pack`.

I'm not advocating for any metatable to be used for two reasons.
First, `__len` is about sequences and this isn't a sequence. This
extra metatable is more about making `#(table.pack(...))` work as you
would *hope* (maybe naively expect, but you'd be wrong) and that is a
conspicuous amount of fanciness in such an austere language.

Second, if sequences are hard for new users, then it's important to
define "new users." If it means "programmer new to Lua who wants to
embed it into their application" then the problem of making it simple
for *their* new users is solved by the programmer extending Lua in a
way that they see fit... or not at all because making sequences
doesn't matter to them. Either way, a C programmer new to Lua can
reason about the language's treatment of varargs and sequences.

If what is meant by "new user" is someone using Lua like Python, then
it's a might be valid issue to resolve. Since this is not Lua's
primary mission, this feature has a high hurdle to overcome and it
would be unfortunate to get it wrong.

-Andrew Starks