Strange experiences with coroutines while using Copas

[Chris Pressey <cpressey@...>]

Hi all,

I'm using Lua 5.0.2, LuaSocket 2.0beta3 w/Compat-5.1r4, and Copas 1.0,
in an environment similar to Xavante.

Am I right in guessing that it's not recommended to use coroutines in
code that is running under copas.addserver()?

Basically, I have a server that reads some stuff from a socket.  It then
creates a coroutine for later use.  The coroutine reads more stuff from
the socket, and yields it in small chunks.  Something like:

  c = coroutine.create(function()
    repeat
      local chunk = skt:receive(chunk_size) -- 'skt' has been
                                            -- copas.wrap()'ed
      coroutine.yield(chunk)
      length = length - chunk_size
    until length <= 0
  end)

  ...

  repeat
    success, data = coroutine.resume(c)
    if data then process(data) end
  until not success or not data

But this only "sort of" works.  The behavior I see is that sporadically
the coroutine.resume(c) gives me a userdata value instead of a string.

I *think* this is because my code, which is running under
copas.addserver(), is already a coroutine in the eyes of Copas... and
the skt:receive is already doing a coroutine.yield(), and this is
interfering with my use of it.  So there's a race condition of sorts,
and sometimes my code receives the string that I am yielding, but
sometimes it receives the socket object that Copas is yielding.
Is that close to right?

There's more, though - and it's even stranger :)

I decided that I would rewrite my code as an iterator to try to work
around this problem.  Something like:

  c = function()
    local length = length
    return function()
      if length > 0 then
        local chunk = skt:receive(chunk_size)
        length = length - chunk_size
        return chunk
      end
    end
  end)

  ...

  for data in c() do
    process(data)
  end

Trying this, I get the following somewhat mystifying error message:

  attempt to yield across metamethod/C-call boundary

But if I rewrite the last part as

  local iter = c()
  data = iter()
  while data do
    process(data)
    data = iter()
  end

...it works again, but only "sort of" - it seems to still have some kind
of race condition with something inside Copas, since it doesn't receive
all the chunks that should be sent, and it sometimes gets data chunks of
unexpected sizes, with no consistent pattern that I can discern.

Thanks in advance for any insight into these oddities.

-Chris