Re: explicitly defining table length + _ARG proposal

[Patrick Donnelly <batrick@...>]

On Tue, Jun 22, 2010 at 12:31 AM, David Manura <dm.lua@math2.org> wrote:
> To followup [1], can someone remind me what is the rationale for Lua
> 5.1 tables not storing an explicit table length, which thereby
> prevents trailing nils from being regularly counted in the table
> length?  I'm not suggesting storing length in a regular table key,
> like the Lua 5.0 "n" field which affects things like pairs iteration,
> but rather storing it in some hidden location accessible only via the
> # operator.
>
> My second comment is that Patrick mentioned something that made me
> think why not support the following:
>
>  function f(...)
>    for i=1,#_ARG do print(_ARG[i]) end
>    g(...)  -- or g(unpack(_ARG)) or maybe g(_ARG[*])
>  end
>
> where _ARG is a specially defined variable lexical to the current
> function with `...` params and containing the values in `...`.  It is
> "specially defined" in the manner of self and _ENV.  It is therefore
> not too unlike the `arg` variable in Lua 5.0 except for a couple
> differences.  Formally, _ARG is a userdata or table with __index and
> __len operators that serve the purpose of `select` and with the __len
> operator including trailing nils in its count.  _ARG may also be
> mutable, allowing things like `_ARG[i] = _ARG[i] or 0`.  Finally,
> there is an important performance optimization: if the function does
> not do things like store `_ARG` in another variable, pass it to
> another function, add keys to it, or use it as an upvalue, then `_ARG`
> can be optimized away by the compiler and doesn't actually need to be
> constructed in memory: #_ARG and _ARGV[i] merely render as opcodes in
> the VM.

(For those curious, there was discussion on IRC about this. I brought
up using _VARARGLEN or something like that as a regular local that is
set to the length of the vararg list. One can take this a step farther
and use _ARG as David as done.)

The idea has a certain appeal but there are many potential problems. I
have not found a satisfactory way to handle passing _ARG to other
functions. I thought maybe introducing a hidden type where _ARG is
actually the running function's variable argument array (luaV_gettable
and luaV_getlen would handle it appropriately) but one quickly
realizes it is too unwieldy when used in other scenarios. How do you
handle a child closure using the _ARG "upvalue"? At this point it no
longer looks like an elegant solution to me. Unless someone has a way
to make it work?

Taking the idea of a hidden type further, I figure we could make it a
formal Tuple. It can be closed in the same way as an upvalue. This
would solve most problems I think.

-- 
- Patrick Donnelly