We don't really need NIL_IN_DICTIONARY, we only need NIL_IN_ARRAY. And the "array border" (or "high water mark") is the right word to solve the problem.
Here is one more (very similar) suggestion for 5.4 which don't require new syntax.
Every Lua table must store 8-byte integer "high water mark" (biggest non-zero positive integer key assigned).
User can read it with function table.hwm(t)
local t = {nil, nil}
assert(table.hwm(t) == 2)
t[math.pi] = 42
assert(table.hwm(t) == 2)
t[5] = 42
assert(table.hwm(t) == 5) t[5] = nil
assert(table.hwm(t) == 5) t[100] = nil
assert(table.hwm(t) == 100)
As you see, the logic of hwm(t) is completely independent of #t.
We have both traditional length #t and array-ish length hwm(t) simultaneously, that solves our problem with nils in arrays.
How arrays with nils should be traversed in 5.4:
for k = 1, table.hwm(array) do ... end
Pros:
1) No broken code. All 5.3 code will run in 5.4 unmodified.
2) No new syntax, so all 5.4 code will have correct syntax from 5.3 point of view.
3) It is possible to write code which runs on both 5.3 and 5.4, but only for nil-less arrays which were not shrinked. Just insert the following line at the beginning: table.hwm = table.hwm or function (t) return #t end
4) pair() will iterate only through non-nils values, as expected. 5) Only non-nils values are stored in the memory.
Cons:
1) hwm(t) is more verbose than #t
2) As previously, #t is nonsensical for arrays with nils, and t[#t+1] doesn't work for arrays with nils. You should write t[hwm(t)+1] 3) No new array type, so arrays don't have specific metatable assigned.
4) 8 bytes more for every table storage. (How many bytes empty table takes?)
Questions:
1) Is table.sethwm(t, newhwm) needed?
2) Should ipairs() use table.hwm() internally? Is new function hpairs() needed?
