Re: coroutine vs threads

[Florian Weimer <fw@...>]

* Tony Finch:

> Florian Weimer <fw@deneb.enyo.de> wrote:
>>
>> This description is already heavily titled to an event-based approach.
>> You don't need coroutines at all for that.
>
> Actually coroutines mesh extremely well with event-based programming.
> Instead of a chain of event handlers you just write straight-line code.
> The central event loop just resumes the relevant coroutines according to
> the events that arrive. You can probably even use a coroutine-oblivious
> core if you use coroutine.wrap.

It's certainly true that coroutines can transform straight-line code
into event sources and receivers.  But there are some thorny aspects
which (to my knowledge) no one has solved satisfactorily.  For
instance, suppose that you've got an internal iterator:

function foreach(t, f)
   local k, v
   while true do
      k, v = next(t, k)
      if k == nil then
	 break
      end
      f(k, v)
   end
end

You can translate that to an external iterator which produces a stream
of events (suitable for use in for loops) with a coroutine:

function iforeach(t)
   return coroutine.wrap(function()
			    foreach(t, coroutine.yield)
			 end)
end

In this example, iforeach pretty much behaves as the original next
function.  But suppose that foreach instead access some external,
non-Lua resource, perhaps using code like this:

function foreach(relation, f)
   local txn = transaction.begin()
   local resultset = txn.query(relation)
   while resultset.next() do
      f(resultset.row())
   end
   resultset.close()
   txn.commit()
end

iforeach will work as before.  But if it is an loop such as:

for k, v in iforeach(relation) do
   if v == 17 then
      break
   end
end

then the transaction will be left around.  In an event-based system,
this will happen, too, but it is a case you have to deal with
explicitly (along with many others, which makes this style a bit
tedious).  In a thread-based system, threads are part of the root set,
even if they are sleeping for some reason, so the code that is
currently executing that for loop cannot be garbage-collected, but
suspended coroutine are subject to garbage collection, so this is
another source of state management issues.  The code transformation
offered by coroutines is powerful, but what it can do about external
resources is necessarily limited.

If I recall, these issues have been observed in actual web frameworks
using the coroutine style, so it's not just a theoretical concern.