What about adding an explicit length field to the table?
If the table has not got a __len metamethod (at the time the metatable
is set), the length field is the maximum of all numeric keys which
have been used to store into the table (including nil values). If
there is a __len metamethod, this field is left unchanged. The
standard library contains rawsetlen() and rawgetlen() functions. The
# operator calls rawgetlen() or the __len metamethod (if available).
table.remove() is adjusted accordingly, so that it decrements the raw
length (in addition to storing nil). Just storing nil does not update
the raw length.
table.unpack(t) returns #t results. table.pack() would not be added
because {...} does the right thing automatically.
ipairs() checks whether table has got a __len metamethod. If it
exists, it is called and the iteration goes up to the value it returns
(this needs consing). Without a __len metamethod, a pre-cooked
iterator which uses rawgetlen() internally is used.
The question is whether it's worth wasting a value object in the table
to store the raw length. But 5.1's nil-terminated arrays appear to be
the aspect of Lua which is most difficult to understand, judging by
mailing list traffic. I think the proposal above would fix this for
good.
