Re: A Batteries API specification (was Re: LuaDEAL - Lua DEad Alive Libraries)

[Sean Conner <sean@...>]

It was thus said that the Great pocomane once stated:
> On Tue, Jan 28, 2020 at 1:35 AM Sean Conner <sean@conman.org> wrote:
> >   I don't have Windows, so I can't say for sure how Windows IOCP works, but
> > I'm hoping they can fit into the scheme I've outlined above.  Thoughts?
> > Comments?  Criticisms?
> >
> >   -spc
> 
> I want to clarify a point: is this API proposed as a "Generic event"
> API (not just a "File/socket" one)? 

  Generally under POSIX, if it has a file handle, it can be used for event
processing.  More and more modern POSIX systems are making non-file objects
available via file descriptors (like signals under Linux).  Right now, I"m
aiming more for a "widely supported event processing" model, which now is
mostly file-related.

> I think that such API is crucial
> for the modularity, i.e. if I will write a C module that interact with
> a USB gamepad, I would like to put "Handler objects" in the same Set
> to monitor both gamepad and sockets contemporary.

  I know my current event library [2] can handle any file descriptor,
sockets, files, devices.  

> Without this, any modularity is illusionary.
> 
> I am not discussing the implementation here, if there is the need for
> a "Fake file descriptor" internally, it can be done [1]. In this
> spirit I would substitute "File descriptor" and "_tofd" with something
> else. 

  I'm open to suggestions.  I don't like "_tofd" myself, but I'm not sure
what a better name would be.  

> I propose this also because the API should make sense in systems
> that do not use the file descriptor for event notification too, like
> windows.

  Agreed.

> Another comment: I have some doubt about the resulting event list.
> Probably events from different sources will be dispatched to different
> code, so the following format could be better (practically a mix of
> your own and the luaposix one):
> {
>   [objA] = {
>     read = true,
>     write = false,
>    ... -- all field always present
>   }
>   ... -- Object field is missing if there is no notification for it
> }

  I've found the propsed method (which is basically what I'm doing with my
current API) to be perfectly fine, and the Lua value I pass in is usually
(okay, always) a function.  For instance, to accept connections via a TCP
socket:

	function listens(sock,mainf)
	  nfl.SOCKETS:insert(sock,'r',function() -- val here is a closure
	    local conn,remote,err = sock:accept()
	
	    if not conn then
	      syslog('error',"sock:accept() = %s",errno[err])
	      return
	    end
	
	    conn.nonblock = true
	    local ios,packet_handler = create_handler(conn,remote)
	    ios.__co = nfl.spawn(mainf,ios)
	    nfl.SOCKETS:insert(conn,'r',packet_handler) -- packet_handler is also a closure
	  end)
	
	  return sock
	end

  The function packet_handler deals with a normal two-way TCP socket, for
example (sans a bunch of unrelated details):

	-- conn is the socket
	-- remote is the remote address

	function create_handler(conn,remote)
	  local ios = mkios() -- this acts like a file: obj of Lua
	  return ios,function(event)
	    if event.read then
	      local _,packet,err = conn:recv()
	      -- deal with data
	    end

	    if event.write then
	      local bytes,err = conn:send(nil,buffer) -- don't need a remote address here, so it's nil
	      -- deal with partial writes, etc.
	    end
	  end
	end

  So I can easily deal with different events on different objects easily. 
In my gopher client for instance, I use the same event driver (even if it
was originally written with network services in mind) to deal with keyboard
input, so I can cancel network requests and such:

	  nfl.SOCKETS:insert(tty,"r",function()
	    syslog('debug',"attempting to cancel")
	    local c = tty.readchar()
	    if c == 'c_C' then
	      cancel()
	    end
	  end)

  The top level driver is written with this functionality in mind (example
sans a bunch of unrelated details):

	events = nfl.SOCKETS:events()
	for _,event in ipairs(events) do
	  event.obj(event)
	end

> Moreover, I think that keeping it simple is more valuable than your
> trick to substitute an optional var to Obj. If a user code really need
> it, it can keep the map in a table by its own.

  Honestly, I didn't find the current implementation all that difficult, and
I did it four different times (select(), poll(), epoll() and kqueue()).  The
mechanism would have to exist *anyway* since what's referenced isn't the
underlying operating system file descriptor [3] but the higher level Lua
abstraction.  The gory details can be seen here [2].

> [1] However, probably every lua-side standard specification will need
> some c-side specification. And probably, some internal details have to
> be exposed there.

  What did you have in mind?

  -spc

[2]	https://github.com/spc476/lua-conmanorg/blob/master/src/pollset.c

[3]	I know, I know, that's POSIX terminology, but I have no better at
	this time.