Re: new "empty" value/type in Lua?

On Jun 28, 2013, at 2:42 AM, Philipp Janda <siffiejoe@gmx.net> wrote:

There already is a level-3 solution to your problem: Have a look at `table.pack` which supports arrays containing nils.

Table.pack() creates a Lua table that (as the docs state) is *not* an array if the list contains nil values.

In your case this probably is unnecessary anyway, because the user usually knows how many values to expect from a database operation -- he/she has written the SQL statement after all …

No, this is not true. This is a generic library that must pack SQL results .. it has no inherent knowledge of the schema or intent of the user (and nor should it). In such a library it is FAR better than it does NOT make assumptions about the statement or schema.

Then there are other implications to consider:
Should `local x` contain nil or empty? Should `table.pack( nil, nil )` translate the nils into empties or not? What about `{ ... }`? Should `table.unpack( t )` translate empties into nils? Should accessing an empty field invoke __(new)index metamethods? `t[ 1 ] or 0` won't work anymore. You would need two conditions to check for valid table elements. There will be sparse arrays where you *don't* want to use empty anyway, e.g.: `{ n=1000, [499]=1, [500]=2, [501]=1 }` ...

I don't see any of these as very hard to answer:

-- "local x" should initialize to nil as it currently does; I see no reason to change the basic language semantics.

-- table.pack(<anything>) should behave as expected, so table.pack(nil, nil) yields {nil, nil} while table.pack(empty, empty) yields {empty, empty}

-- Same for {…}

-- table.unpacl() should not translate nil to empty

"empty" has no magic (unlike nil), for example to my mind "empty" when used in a boolean context should return true like any other value except false/nil. "empty" is just a value that is totally ordinary EXCEPT it's NOT equal to any other Lua value except itself. So of course it doesn't modify __newindex() in any special way. In fact, what you have highlighted is just how special "nil" is in Lua, and how it is overloaded in odd ways. For example: "x = nil" has different behaviors for globals and locals. Yes, I completely understand why, but the fact is it's different.

And again I'll point out that Lua arrays DO behave oddly when faced with nil. I have no problems with this, I've been developing for so long in so many languages I'm used to god knows how many quirks, but quirks they remain. Stop and think about it:

a = {1,2,3} -- it's an array

a[4] = "hello" -- still an array

a[2] = 1000 -- STILL an array

a[4] = nil -- STILL an array

a[2] = nil -- oops .. not an array any more

a[5] = 99 -- not an array

a[2] = 1 -- Hey! .. I'm an array again

For a junior developer, it's even harder to see when hidden behind locals:

local x, y = 10

a = {1,2,3} -- It's an array

a[2] = y -- Not an array any more

a[2] = x -- Wait .. I'm an array again

Imagine a Java/C++/C# collection behaving like that. I'd be scratching my head a bit at such a thing. (OK, so Java/C++/C# collections have a LOT of other problems, but two wrongs don't make a right! <grin>).

There are plenty of cases where being able to "mark" an array entry as done/dead/empty etc is useful as part of an algorithm. In many cases you can do that using a sentinel value that is outside the normal range of expected array types and/or values, but it's all very domain specific (can I use an empty string? -1? false? auxiliary state?). I'm basically arguing that "empty" allows you to do this in a nice, clean, self-documenting, portable manner. I don't really see anything heretical in that. To be honest, the most heretical thing is that you would introduce a new keyword, and THAT can of course break existing code.

--Tim