Re: lists with nil play nice for Lua 5.2

[John Belmonte <john@...>]

I'd like to step back from the argument of whether nils belong in lists
or not.  Rather I'll point out some general, but related, gaps in Lua's
extensibility and orthogonality-- hopefully there will be less
disagreement about the need to address these.

1)  no hook for length operator (#) on table or string

This is a wart in Lua's extensibility, and often results in complex and
inefficient workarounds.  If the lack of __len metamethod support on
these types is solely an efficiency concern, why not use the pattern of
offering a raw version of the operation?  I.e. add lua_rawobjlen to the
C API-- the precedent is already set by lua_rawequal/get/geti and
related metamethods.


2) loss of sequence information in table constructors

Table constructors drop vital length information about their sequence
portion.  It can be recovered with the length operator, but only when
the sequence does not contain nil values.  This is in contrast with
function argument sequences, where the length is available via
select('#', ...), regardless of nil's.

The core has the length information when it creates the table.  The
issue is how to present it to the user in a way that is efficient,
elegant, and doesn't give the impression that the length will be updated
on subsequent mutation of the table.

One idea which has flaws but meets the requirements is to have table
constructors yield a table + length pair, e.g.:

  > print {'apple', 'pear', nil}
  table: 0x807a5e0        3
  > print {assert(true, nil, 'foo')}
  table: 0x807a5e0        3

For the majority of table constructor uses this would be backwards
compatible.  The gotcha is when a list constructor is used as the last
argument in a list or function argument sequence.

  list = {5, 10}             -- OK
  f({5, 10}, 'hello')        -- OK
  nested = {'foo', {5, 10}}  -- extra list element may be unexpected
  f2('foo', {5, 10})         -- extra arg may by unexpected
  f2('foo', ({5, 10}))       -- OK

Less intrusive to existing behavior would be to say, store the
constructor length by weak key of the table in a global table.  Needing
access to any global for such basic information is ugly though.