Re: Upcoming changes in Lua 5.4

On Mon, Jan 8, 2018 at 3:36 PM, Roberto Ierusalimschy wrote:

> Won't this add significant overhead for the table allocations vs just
> leaving the vargs on the stack?

That was the motivation for stack varargs. But "significant" is quite
relative. Moreover, stack varargs add an overhead (a quite small one, but
it is not zero) to all functions, even those not varargs. We do not see
vararg functions in critical paths often. Several (most?) vararg functions
have to create tables (or pay the price of 'select') anyway.

More important, table is the bread-and-butter of Lua. If we start to
demonise them, we are left with little else.

Just tested Lua from GutHub.

Remark #1:
"Microsoft Visual Studio 2010 x64 tools" displays 4 warnings during compilation of Lua 5.4.0:
lapi.c(995) : warning C4244: '=' : conversion from '__int64' to 'int', possible loss of data
ldo.c(676) : warning C4244: '=' : conversion from '__int64' to 'int', possible loss of data
ltable.c(382) : warning C4334: '<<' : result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?)
lvm.c(1501) : warning C4244: '=' : conversion from '__int64' to 'int', possible loss of data

Remark #2:
A bug
Lua 5.4.0 Copyright (C) 1994-2017 Lua.org, PUC-Rio
> function f(x, ...) end
> f()
not enough memory

Remark #3:
The syntax f(x, y, z, ...=arg) is not very convenient.
In real-life use cases sometimes we want to avoid creating a table,
and sometimes to include both named and dots arguments in a table.

-- In this example the function    insert(obj, elem1, elem2, ...)
-- sometimes should be treated as insert(obj, elem1, elem2)
-- and sometimes as                insert(obj, <<elem1, elem2, ...>>)
-- but never as                    insert(obj, elem1, elem2, <<...>>)
------------------------------------------------------------------------
local unpack = table.unpack

-- insert elements into some object
local function insert(obj, elem1, elem2, ...)
   -- this function may have 1..inf actual arguments
   if not elem2 and elem1 then
      -- user invoked this function to insert single element
      -- this is the most frequent usage of this function
      -- for the sake of performance we avoid to create a table
      ----------------------------------------------------------------
      -- optimized code for inserting single element (elem1)
      ----------------------------------------------------------------
   else
      -- user invoked this function to insert multiple elements
      -- we are creating a table containing named arguments and dots
      local elems = {elem1, elem2, ...}
      for j = 1, #elems % 8 do
         insert(obj, elems[j]) -- recursive call to insert one element
      end
      for j = #elems % 8 + 1, #elems, 8 do
         local e1, e2, e3, e4, e5, e6, e7, e8 = unpack(elems, j, j + 7)
         ---------------------------------------------------------------
         -- inserting full block of eight elements
         -- (it is faster than inserting 8 elements one by one)
         ---------------------------------------------------------------
      end
   end
end
------------------------------------------------------------------------

Question #4:
Will we have an ability in Lua 5.4 to include dots along with
some of named arguments in an array?
For example,
   insert(obj, {elem1, elem2, ...}=elements)

Remark #5:
Unfortunately, the "bread-and-butter of Lua" is quite expensive.

The file "benchmark.lua" is here: https://pastebin.com/dhCqcMgE
The output:

Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio
> dofile[[..\benchmark.lua]]
6.9270000000001
> dofile[[..\benchmark.lua]]
7.0349999999999
> dofile[[..\benchmark.lua]]
7.02

Lua 5.4.0 Copyright (C) 1994-2017 Lua.org, PUC-Rio
> dofile[[..\benchmark.lua]]
21.887
> dofile[[..\benchmark.lua]]
21.668
> dofile[[..\benchmark.lua]]
22.106

Question #6:
Can expensive operation of creating a table in "f(x, y, z, ...=arg)"
be deferred until the moment of actual access to "arg"?
So that some branches of execution (which do not use arg) will not suffer of
a performance impact due to creating a table they don't need?

-- Egor