lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


5.2 – Variable Number of Arguments

   CALL            PARAMETERS
    g(3)             a=3, b=nil, arg={n=0}
    g(3, 4)          a=3, b=4, arg={n=0}
    g(3, 4, 5, 8)    a=3, b=4, arg={5, 8; n=2}

Using those regular parameters, the definition of select is straightforward:

    function select (n, ...)
      return arg[n]
This "definition" does not reflect the reality. Actually the select() function does not just returns arg[n], it returns all arguments starting at index n up to the number of arguments in arg, i.e. it returns arg[n], arg[n+1], ... arg[#arg]

Note that that page of documentation uses an usual non-sense notation to describe the content of the local parameter named "arg" (implicitly declared in all functions):
arg={5, 8; n=2}
This notation uses a semicolon in a table constructor, a syntax that does not exist in Lua. It may suggest that this semicolon indicates that the key/value pair n=2 is not stored in the table itself, but in its metatable.

Such observation cannot be made, but if it does, then any declared function that explicitly accesses to its "arg" parameter could have to create (on function entry) a new table indexing all upvalues (including nil values), and another table to store the key/value pair with an implicit key name 'n' (here ['n'] = 2) containing the effective number of upvalues (including nil's).

Such creation of two tables would be very inefficient (stressing the garbage collector), so I suspect that instead the "arg" type is not a true "table" but behaves "like a table" in user's Lua code.

But the Lua parser and compiler knows that it is not a real table, and instead it just uses the internal upvalues array directly (i.e. as the integer-indexed array part of any table, whereas upvalues do not have any hashed part containing arbitrary keys but allow storing nils directly at any integer index in the valid range, and that the "n" notation used here refers to the maximum index allocated in this integer-indexed array.

[==[ Sidenote:
Despite this, I like the notation used, with the semicolon: if it was available in Lua syntax for table constructors, to attach a metatable after the normal enumerated keys, we could avoid the use of "setmetatable" in a separate statement, and could create tables with their metatable directly within expressions.
We could as well attach a metatable to the metatable, by adding another semicolon followed by another list of key/value pairs.

But the doc page is still incoherent in that case as it shows another notation:
which is incorrect and would suggest that the key/value pair is stored actually in the arg table directly, or is accessible with:
In reality that hidden pseudo-field is accessible via
as if the pseudo-table arg had a metatable not just "containing" the key/value pair with pseudo-key name "n" (via an "__index" metamethod), but also a metamethod "__newindex" which generates an error (we cannot reassign any value to any arg[index], where index is not in the existing range 1 to #arg)

The Lua doc has hints about the builtin use of pseudo-tables (or "weak table") in section 17.2:

17.2 – Object Attributes

Lua itself uses this technique to keep the size of tables used as arrays. As we will see later, the table library offers a function to set the size of an array and another to get this size. When you set the size of an array, Lua stores this size in a private weak table, where the index is the array itself and the value is its size.

But I doubt this is really a "weak table" (as defined in previous section, it would mean that the arg "pseudo-table" would have a "__mode" key set with a non nil value to make it "weak".

So to conclude, the section 5.2 (which is the only one describing the implicit parameter "arg") is very confusive.

It does not refers to the correct feature of Lua, which is using "upvalues" as a real array with a fixed size (accessible as "#arg"). "arg" has a special type
So Lua really has a type for arrays indexed by positive integers only.

My opinion is that this special subtype of table should be exposed as a true type, or as standard properties of the table type (notably that it allows only integer keys, and that if effective size "#t" is unmutable). It would still not necessarily be an array and may still have an hashed part for sparse arrays (notably if "#t" is large and most keys are set to nil values).

We should have a standard library call to create or transform a table into an array with these constraints: when transforming a table with a library call, it would check that the hashed part contains only positive integer keys (for keys outside the integer-indexed array part), and then could either  (with a call flag?) drop the offending keys or return an error. If the table has a metatable, it may eventually be kept, but it is not necessary to get the best performance offered by array.

It could also offer a way to recompact the array between the integer-indexed part and the hashed part, so that the first part would have a minimal rate of non-nil values (e.g. at least 50%), and all other positive keys would be hashed. The internal hashing function for positive integers can be much simpler than for arbitrary types.

Note that arrays currently used by upvalues allocated on the stack for function calls have no minimal rate of non-nil values, and they are small anyway. We currently have a limit somewhere between 50 and 242 due to how Lua generates its bytecode. Lua could extend this limit by allowing upvalues for function calls (or any tables transformed into arrays) to be stored outside of the call stack (using dynamic memory, or paging it onto a virtual memory cache, backed by external storage with optional data compression), leaving only an object reference on the stack.