Re: table.concat is inefficient for very long strings

[Francisco Olarte <folarte@...>]

HI Egor:

On Fri, 22 Jul 2022 at 10:34, Egor Skriptunoff
<egor.skriptunoff@gmail.com> wrote:
> On Thu, Jul 21, 2022 at 9:05 PM Gé Weijers wrote:
>> local N <const> = 0x1 << 30
>> local index_count = 0
>> local meta = {
>>     __index = function(meta, key)
>>         index_count = index_count + 1
>>         if index_count <= N then
>>             return "a"
>>         else
>>             return "bb"
>>         end
>>     end,
>> }
>> local t = setmetatable({}, meta)
>> print(table.concat(t, "", 1, N))

> Never do this in a production code.  Part 2.

I do not think the OP does that in production code but...
... you may have a table which varies like....

local hardware_switch_meta = {
     __index = function(meta, key)
           if (not math.tointeger(key) or key<1 or
key>hardware_switch_count()) then return nil end
           return hardware_switch_status(key-1) and "ON" or "off"
    end,
}

Error checking omitted, assume hardware_switch_status returns a
boolean CURRENT status of a real switch.

> Again, you're relying on unspecified behavior here.

He is not relying on anything, just making a convoluted example.

> Why do you think table.concat must read each table element only once?
> The Lua manual never promised that.

If I had done that ( the hardware switch example ) I would not assume
single reads, but I would expect somehow correct behaviour, i.e., no
crashes and a string ( potentially meaningless ) returned on a finite
time. Otherwise I would expect a note on the manual.

.. insightful evaluation order comments snipped...

> I have started this thread with a description of a real problem every programmer may stumble upon:
> when your Lua script utilizes a better part of your RAM
> some goals are achievable with the concatenation operator but not with table.concat.
> Why do you reply to me with a contrived code that does not solve any problem?
> I can't imagine what your code might be useful for in practice.

Not the OP, but I think what he is trying to point is simple tricks
for avoiding reallocs may be dangerous. I.e., if you make one pass,
allocate, make a second pass copying, you may encounter yourself in a
buffer overrun situation, so table.concat must be coded in a way which
tolerates degenerated table behaviour w/o crashing. Easy-peasy to do,
but worth remembering.

Francisco Olarte.


>