Re: [ANN] article: Structured concurrency and Lua

[Sean Conner <sean@...>]

It was thus said that the Great Thijs Schreijer once stated:
> On 2 Oct 2022, at 15:27, John Belmonte <john@neggie.net<mailto:john@neggie.net>> wrote:
> 
> > I've posted part 3, which fleshes out nursery cancellation:
> > 
> >  https://gist.github.com/belm0/20bc069ca676fcfc591f29930069f788
> 
> I’ve followed the discussions here and find it really interesting. A major
> drawback is the missing ability of Lua to run “nested” coroutines without
> getting into nasty problems.
> 
> For example; if you run this on top of Copas, and a Trio task would call a
> Copas socket function (read/write/connect) it would do a Copas yield to
> the Trio scheduler (meaning it would pass the Copas socket and request
> (read/write) to Trio, which would not be expecting those.

  I was able to implement a Trio type library for my own driver [2] that
doesn't have any of the issues mentioned.

> Even if you use the tagged coroutines [1] it would probably mean that a
> single Trio task calling a read/write op, would mean the entire Trio
> scheduler would get paused.
> 
> So do you think it would be possible to implement this on top of an
> existing scheduler? What primitive functions would be required to make it
> work? In an abstract interface with things like;
> 
> - runtask(func)
> - sleep(seconds)
> - sleep_forever()
> - wakeup(coro)
> - canceltask(coro)

  I have the following API for my "nursery" module:

	nfl = require "org.conman.nfl"
	nursery = require "..." -- not published yet

	n = nursery(timeout)
		Create a nursery.  The parameter will cancel all tasks after
		timeout seconds.  If no paremeter given, no timeout will
		occur (this uses nfl.timeout() to control it).

	n:spawn(f,...)
		Create a coroutine with f, using ... as the arguments.  This
		calls into my nfl.spawn() function.

	n:wait()
		Wait for all spawned coroutines to finish.  The __close()
		metamethod on the nursery is this function.

	n:wait_any()
		Wait for any spawned coroutine to finish; the rest are
		cancelled (that is, removed from my nfl coroutine reference
		list, and coroutine.close() called on them).

Each of these routines also return the nursery object, so calls can be
chained.  An example from some working code I have:

	local function get_quote(state)
	  local conn = tcp.connect(QOTDSERVER,'qotd')
	  if conn then
	    for line in conn:lines() do
	      table.insert(state,line)
	    end
	    state.done = true
	    conn:close()
	  end
	end

	-- get_gopher() and get_gemini are similar, except that
	-- get_gemini() uses TLS instead of bare TCP.

	local function server(ios) -- ios is "Input Output Stream"
	  local gemini = {}
	  local gopher = {}
	  local quote  = {}

	  do
	    local _ <close> = nursery()
		:spawn(get_gemini,gemini)
		:spawn(get_gopher,gopher)
		:spawn(get_quote,quote)
	  end -- wait for all tasks to finish

	  -- rest of code
	end

	tcp.listen("127.0.0.1",1717,server) -- run server() when connection is made
	nfl.server_eventloop() -- main driver loop

This will also work:

	nursery()
	  :spawn(get_gemini,gemini)
	  :spawn(get_gopher,gopher)
	  :spawn(get_quote,quote)
	  :wait()

if you don't have Lua 5.4 or don't want to bother with the __close() stuff. 
My nursery module is less than 100 lines of code currently, and wasn't hard
to write.  I don't yet have an API for general cancellation of routines
though.

  -spc
	
> [1] see https://github.com/mascarenhas/taggedcoro and https://github.com/saucisson/lua-coronest

[2]	https://github.com/spc476/lua-conmanorg/blob/master/lua/nfl.lua