Coroutines & Iterators

[MichaelL@...]

I have a suggestion for Lua5. I'd like to suggest that some syntactic sugar
would offer a different--and potentially better--mechanism for iteration
than couroutines. Bear with me for a moment while I explain!

The idea is to have slightly different sytax for anonymous functions--and
to have special parsing for such anonymous functions when they follow a
function call. (This is similar to what Ruby does, and what Ruby does is
similar to what Smalltalk does. Well, sort of. Not really!)

That means that instead of this:

  x:somefunction( x, y, function (i) print(i) end )

You'd have something like this:

  x:somefunction( x, y ) do |i|
    print(i)
  end

(NOTE: I'm advocating the idea, not this specific syntax. I'm hijacking the
syntax for "chunks" here, and I don't know that that will *really* work.)

What would you do with such syntax? Here are some examples:

  x = SomeTableWithMetatableFunctions{ "a", "b", "c" }

  x:each() do |i|
    print (i)
  end
    => prints "a", "b", and "c"

  x:find() do |i|
    if i > "a" then return true else return false end
  end
    => returns "b"

  x:findAll() do |i|
    if i > "a" then return true else return false end
  end
    => returns { "b", "c" }

  x:rejectAll() do |i|
    if i > "a" then return true else return false end
  end
    => returns { "a" }

This syntax is also useful for things that don't take parameters:

  withOutputToFile( "test.txt" ) do
    -- do some write stuff
  end

  withTransaction() do
    -- do some database stuff
  end

Again, it's just syntactic sugar--but so is "x:somefunc( ... )" and
"somefunc{ ... }" and "{ x = "zzz", ... }. Lua's big on syntactic sugar!

The iterators above--that is, the functions I've given as examples--would
have an extra argument, a function called "yield." (This is rather like the
x:somefunc( ... ) syntax adding a parameter called "self.")

So:

  function x:each()
    for i, v in self do
      yield(v)
    end
  end

  function x:find()
    for i, v in self do
      if yield(v) then
        return v
      end
    end
    return nil
  end

  function x:findAll()
    local result = {}
    for i, v in self do
      if yield(v) then
        tinsert( result, v )
      end
    end
    return result
  end

  function x:withOutputToFile( file )
    writeto( file )
    yield()
    writeto()
  end

  function x:withTransaction( file )
    startTransaction()
    yield()
    commitTransaction()
  end

I haven't bothered with some niceties (like error-handling) but you get the
idea. (Actually, error-handling makes the idea more compelling--because it
allows you to, for example, say "always reset the writeto() file" or "if
there's an error, rollback the transaction." The person who *creates*
"withTransaction()" can make that work as expected--rather than everyone
who *uses* startTransaction().)

An aside: Ruby has a concept of "Modules," which are collections of
functions that can be mixed into a class. In one module Ruby defines many
functions--like find() and findAll()--that are all written in terms of each
(). That means that once you implement each(), you can get a host of other
useful functions for free if you choose to mix them in. Here are examples
of alternate implementations:

  function x:find()
    self:each() do |v|
      if yield(v) then
        return v
      end
    end
    return nil
  end

  function x:findAll()
    local result = {}
    self:each() do |v|
      if yield(v) then
        tinsert( result, v )
      end
    end
    return result
  end

Why is this better?

It seems like Lua5 is headed toward using coroutines for iterators.
Coroutines are, by comparison, relatively heavy-weight objects. Indeed,
coroutines effectively have their own stack and state and are generally
allocated from the heap.

Everything I've shown above works from the current stack. Again, this is
simply syntactic sugar for something that's already possible in Lua4: call
a function, and let that function call a function which is passed in. Very
simple--and very quick. In more ways than one.

The new "for" syntax in Lua5 can *almost* do this. There are three issues:

 * first, as I said, it seems to assume coroutines, which are relatively
heavy objects;
 * second, it assumes that the coroutine will produce at least one var;
 * third, it implies (by using the suggestive "for ... in ..." phrasing)
iteration--but you don't always want to iterate, sometimes you just want to
"wrap".

Coroutines are good--I'm not advocating their removal--but I think they're
overkill for iteration. And, what's more, I think the "syntactic sugar"
would help to encourage a style of programming that was intended with the
original foreach() and foreachi() functions.

Thoughts?