Re: reusing objects for tight iteration loops

[Daurnimator <quae@...>]

On 31 January 2014 11:54, Josh Haberman <jhaberman@gmail.com> wrote:

This avoids the per-row allocation cost. But this doesn't translate to
Lua very well:

  local items = {}
  -- Suppose fast_iterator() is optimized to return the same
  -- row over and over, but with the fields overwritten with new values.
  for row in fast_iterator(data_stream) do
    -- Oops! My table "items" ends up with n of the same value,
    -- with all fields set to the values of the last row.
    items[#items + 1] = row
  end

Easiest (technical) solution:

Just document that row changes, if you want a copy add a row:clone() 

If references can escape then you inherently can't reuse row.

We can try to work around this, but things are still dicey if the row
has sub-objects:

  local row = FastIteratorObject(iterator)
  while row:next_object()
    -- Now it's clear that "next_object()" mutates the row to be the
    -- contents of the next row.
    -- But imagine that row:next_object() can change row.bar.baz:
    do_something(row.bar)
  end

So I was just wondering if anyone had any out-of-the-box ideas about
mitigating this. One idea I had was to make the inner function a
string:

  fast_iterate(iterator, "function (row) print(row.foo) end")

This allows me to precisely control the function's environment, so I
can prevent references to "row" from escaping through the global
environment or upvalues. The downside is that then none of the
program's functions or variables will be visible, which limits the
usefulness of this approach pretty severely.

You don't have to go as far as having users pass a function as a string.

You can check if a function has upvalues via lua_getupvalue

==> If it does, then use the non-mutating iterator

 

D