Re: Thoughts on {...} and tbl[nil]

On Mon, Jun 4, 2018 at 4:27 PM, nobody <nobody+lua-list@afra-berlin.de> wrote:

On 2018-06-02 19:46, Andrew Gierth wrote:

"Rodrigo" == Rodrigo Azevedo <rodrigoams@gmail.com> writes:

Rodrigo> NIL_IN_TABLE is good only for 'arrays' types,

This is absolutely not true; I've been finding the lack of nils in
tables _very_ frustrating, and none of my use-cases have been anything
to do with arrays as opposed to more general tables.

Can you (abstractly, briefly) describe a few of them?

(For me, `nil` is a _symbol_ for "there's nothing here", so the approach of actually storing nils seems backwards… and so I'm looking for other solutions. Having a bunch of test problems would be handy. I scanned through the relevant threads I recalled and didn't find any good ones.)

Regards,
nobody

Lua's use of nil is pretty clear and taken in isolation, is simple and efficient.

- It represents mu [1]

- it is false-y (`not not nil == false`) but is not false (`nil ~= false`)

- Is the default value for all un-initialized variables, unused arguments and unused indexes in a tables

- Is generally how you "remove" values from a table (`t[i] = nil`) [2]

In my experience, this is sufficient. For example, representing a sequence where some of the values are unknown is easily accomplished in Lua:

- false

- `.n` field to denote length

- metatables and/or an API to emulate sequences with holes in a way that works for your application.

- make a userdata object that represents your sequence in a way that works for your host application.

- Use a sentinel, such as JSON_NIL or whathaveyou if that works better for your application

So, why is there a desire to "store" `nil` in tables in such a way as to make them count towards the length of a sequence? What drives people to "solve" this "problem"?

I am interested in any reasonable answer to this question. My attempts are:

## Interoperability with external languages and libraries

Because other languages can store NIL/NUL/MU in sequences, there are APIs that may/do require some way to represent it. If the programmer is not familiar with Lua, the particular way to bridge this gap is not obvious.

They may feel like they have to write too much of their own library code or too much meta-programming. They may pre-suppose that their solution will be slow.

How can this problem be solved? Examples in PiL with possible solutions? A white paper on handling sequences/nil in Lua when your API requires it? Does Lua necessarily need to be "fixed" in this case?

## Speed

Lua "sequences" are usually very fast in normal use cases, thanks to VM optimizations. The solutions for making transparent, bulletproof sequences (length works with holes) inside of Lua involves some combination of indirection, function calls / metatables and the Length operator. Speed gains are lost or at least diminished.

One solution is to use a dedicated userdata object. C Programmers can make sequences with their native type and then use Lua's API to provide an interface to this object.

Why isn't this desirable? Too much meta-programming? Not obvious to people unfamiliar with Lua?

# It's not Lua

The thrust of why I wanted to share this was to point out that making Lua more useful often requires the language to change not at all, but for people to understand its design better and to understand the limitations of that design and how to use best use those limitations to your advantage. Or it requires a change to Lua.

If future versions of Lua feature a more complicated concept of mu, only to make it more obvious to people that were not able to see how their project could be accomplished with the simpler way, then that would be sad.

Andrew Starks

[1] http://www.awakin.org/read/view.php?tid=583

[2] but garbage collection happens on its own and so getting rid of allocated objects should still use an explicit `:close()` method.