Re: Issues with the current varargs implementation (Was: Re: inadequate documentation for select and table.unpack)

[Patrick Donnelly <batrick@...>]

On Tue, Jun 21, 2016 at 1:25 PM, Soni L. <fakedme@gmail.com> wrote:
>
>
> On 21/06/16 09:09 AM, Viacheslav Usov wrote:
>>
>> On Mon, Jun 20, 2016 at 9:04 PM, Roberto Ierusalimschy
>> <roberto@inf.puc-rio.br <mailto:roberto@inf.puc-rio.br>> wrote:
>>
>> > That is not true. 'select' can be quite slow for a large number of
>> > parameters (where "large" starts in the order of a few dozen). In
>> > particular, a typical loop in a vararg function using 'select' is quadratic
>> > with the number of arguments, while creating a table and traversing it is
>> > linear.
>>
>> That could be why the manual and PiL are a little shy about this function.
>>
>> Yet, this discussion has revealed that even those familiar with select
>> have very wrong ideas about it. It looks like there is no really lightweight
>> mechanism in the language to access an arbitrary variable in a variadic
>> function. It may be that Lua could benefit from some new syntax for that.
>>
>> Another possibility could be in making {...} very lightweight. That syntax
>> could create a new internal type that is essentially just a maker; and in
>> all the cases when such a value can potentially be leaked from the current
>> function, it gets replaced with a real table. I should say that I am not
>> sufficiently familiar with Lua's internals, so this could be a ridiculous
>> thing to propose :)
>>
>> Cheers,
>> V.
>
> Varargs aren't lightweight. Inserting new varargs requires copying the whole
> varargs array, even for tail calls (however, this could be trivially
> optimized, at least in theory, by treating varargs as a proper stack and
> using the realloc-able end as the first position).
>
> They could be made somewhat lighter by using a linked list of sorts, see
> [1], but that comes with its own set of drawbacks.
>
> [1]: http://lua-users.org/lists/lua-l/2016-06/msg00189.html

I don't think anyone wants to actively pursue making the
implementation of how varargs work now any better. The fact is, the
Lua authors *could* improve vararg performance so that select isn't so
abysmal but ... you're still using select. i.e. no one wants to spend
time optimizing for a function that is widely considered a wart on the
language.

The truth is that using Lua means embracing tables as your only data
structure. So, if you want to **manipulate** your arguments then put
them in a table. Creating new language syntax/semantics for this
ancillary data structure is antithetical to the overall design of Lua.

-- 
Patrick Donnelly