Re: New array type? (was: 'table' as fallback for tables)

[Dibyendu Majumdar <mobile@...>]

On 1 July 2016 at 02:01, Jonathan Goble <jcgoble3@gmail.com> wrote:
> Most languages have two distinct types for sequences and associative
> arrays. Lua has just one type, the table, which is really good at the
> latter, but doesn't do a very good job at the former. The ongoing
> discussion proves that there is a need for a real sequence type
> capable of holding `nil`, but I just don't think there is a good,
> clean way for Lua, as it stands right now, to do that short of
> creating a new type for sequences and deprecating tables for that
> purpose (they would remain useful for associative arrays, of course).
>
> A new type would have many advantages, chief among them elimination of
> the need to store the length as a field in the table or metatable
> (since the length would now be known and tracked by the C internals),
> and elimination of the question of how to set the initial length from
> a constructor (since array constructors probably wouldn't allow
> explicit keys/indices).
>

Hi, in Ravi (Lua 5.3 derivative) I support arrays of integers and numbers.
I have extended the table structure to store additional information.
This includes - a flag to indicate whether the table is regular table, or
number array, or integer array. The length of the array, plus actual array
data. For integer and number arrays I store the data separately to ensure it
is just like a C array.

There are some difficulties in implementing arrays in Lua without breaking
backwards compatibility. First difficulty how to know an array is being
constructed. In Ravi I solve this in two ways:

a) Type annotation allows users' intention to be made clear.
b) I also provide API functions table.numarray() and table.intarray() to return
preconstruced arrays of fixed size.

With standard Lua, assuming that the table internals and VM internals are
enhanced as I have done in Ravi - then the API function approach might work.

The second difficulty is how Lua handles tables, and what NIL means. In the case
of number and int array in Ravi there is no concept of a NIL element.
All elements
are initialized (to user supplied value or zero).

Perhaps the easy solution might be to only support fixed length
arrays, and thereby
allow NILs as valid values.

Finally in Ravi I disallow meta functions __index, __newindex and
__len on arrays
as they do not make sense in the context of an array.

A downside to the Ravi approach is that every access to a table
element has an extra
if condition as we need to check whether the table is a regular table
or an array. But this
cost is negligible, In Ravi of course when type annotations are
available, this cost is
eliminated but that will not be possible in regular Lua.

Regards
Dibyendu