• Subject: Re: Nested iterators, how to best implement?
• From: Nagaev Boris <bnagaev@...>
• Date: Thu, 29 Jan 2015 14:39:07 +0000

On Thu, Jan 29, 2015 at 2:13 PM, Tony Papadimitriou wrote:
Hi all,

I’m looking for the best (?) general way to implement *nested* iterators in Lua (5.3, of course).

For example:  I have an iterator that returns single items when used in a FOR .. IN loop.  Each item is treated somehow but that loop, and the result I want it to become available in another iterator.

Easy, but I do NOT want to store all intermediate results in a table, and then traverse the table again, as the size of the table could be too large or even unknown while the iterator is running (until some condition happens that terminates it).

In Python 3, for example, this would be very simple with each routine using YIELD instead of RETURN, and iterators can be nested to an arbitrary level.  But, in Lua, I can’t seem to find an obvious (memory efficient) solution.  (I would love for Lua to have the same simplicity in this regard.)

Perhaps explicit use of co-routines is one solution, but maybe there is a more immediate way I missed.

Maybe someone can point me to some example.

TIA.

Tony P.

Hello Tony,

you can use a closure. This approach is more complex, then yield'ing.

Example:

-- simple iterator

local data1 = {1, 2, 3}

local function iter(arr)
local it, t, index = ipairs(arr)
return function()
local value
index, value = it(t, index)
return value
end
end

for v in iter(data1) do
print(v) -- prints 1, then 2, then 3
end

-- nested iterator

local data2 = {{4, 5}, {6, 7}}

local function iterNested(arr)
local it, t, index = ipairs(arr)
local it2, t2, index2
return function()
local value
if index2 then
index2, value = it2(t2, index2)
end
if not index2 then
-- start new sub-array
local subarray
index, subarray = it(t, index)
if not subarray then
return nil
end
it2, t2, index2 = ipairs(subarray)
index2, value = it2(t2, index2)
end
return value
end
end

for v in iterNested(data2) do
print(v) -- prints 4, then 5, then 6, then 7
end

Best regards,
Boris Nagaev