Re: Nested iterators, how to best implement?

[Nagaev Boris <bnagaev@...>]

On Thu, Jan 29, 2015 at 2:13 PM, Tony Papadimitriou <tonyp@acm.org> wrote:

Hi all,

 

I’m looking for the best (?) general way to implement *nested* iterators in Lua (5.3, of course).

 

For example:  I have an iterator that returns single items when used in a FOR .. IN loop.  Each item is treated somehow but that loop, and the result I want it to become available in another iterator.

 

Easy, but I do NOT want to store all intermediate results in a table, and then traverse the table again, as the size of the table could be too large or even unknown while the iterator is running (until some condition happens that terminates it).

 

In Python 3, for example, this would be very simple with each routine using YIELD instead of RETURN, and iterators can be nested to an arbitrary level.  But, in Lua, I can’t seem to find an obvious (memory efficient) solution.  (I would love for Lua to have the same simplicity in this regard.)

 

Perhaps explicit use of co-routines is one solution, but maybe there is a more immediate way I missed.

 

Maybe someone can point me to some example.

 

TIA.

 

Tony P.

Hello Tony,

you can use a closure. This approach is more complex, then yield'ing.

Example:

-- simple iterator

local data1 = {1, 2, 3}

local function iter(arr)
    local it, t, index = ipairs(arr)
    return function()
        local value
        index, value = it(t, index)
        return value
    end
end

for v in iter(data1) do
    print(v) -- prints 1, then 2, then 3
end

-- nested iterator

local data2 = {{4, 5}, {6, 7}}

local function iterNested(arr)
    local it, t, index = ipairs(arr)
    local it2, t2, index2
    return function()
        local value
        if index2 then
            index2, value = it2(t2, index2)
        end
        if not index2 then
            -- start new sub-array
            local subarray
            index, subarray = it(t, index)
            if not subarray then
                return nil
            end
            it2, t2, index2 = ipairs(subarray)
            index2, value = it2(t2, index2)
        end
        return value
    end
end

for v in iterNested(data2) do
    print(v) -- prints 4, then 5, then 6, then 7
end

Best regards,
Boris Nagaev