Re: io.popen read and write, again

[Jay Carlson <nop@...>]

On Jul 9, 2014, at 5:27 PM, Sean Conner <sean@conman.org> wrote:

> It was thus said that the Great Jay Carlson once stated:
>> On Feb 26, 2014 10:09 PM, "Sean Conner" <sean@conman.org> wrote:
>>> 
>>>        list = {}
>>>        cmd = U("ls -l") .. U("grep foobar") / "grep-stderr" .. function()
>>>                        for line in io.lines() do
>>>                              table.insert(list,line)
>> 
>> 
>> For maximum fun, use a non-blocking io implementation. You can have
>> arbitrary lua tasklets at any stage of the pipeline, I think.
> 
>  Well, that was all theory, seeing how nobody has actually *implemented*
> what I outlined.  

Yeah. I asked myself, "why not?" and noticed io.lines() wasn't going to be stock.

>  But allowing arbitrary Lua coroutines (is that what you mean by
> "tasklet"?) will be challenging to get right.  

I don't actually know what I mean by "tasklet". :-)

The interesting problem is to allow arbitrary Lua pipeline members to have access to globals. The easy thing to do for pipeline members is fork(), but that breaks the code fragment above. The table named "list" remains empty. 

As you're about to point out, this is not a problem in this particular example. It isn't for any pipeline with exactly one Lua function source or one Lua function sink. You can't popen a process for both reading and writing, so you can't build general pipelines with Lua intermediates with just popen().

The big reason you can't popen a process for both reading and writing is probable deadlock with blocking I/O primitives. It's not a problem with independent processes hooked together by pipe(), and that's because Unix's design is just magic. You don't need non-blocking I/O and select() until you get to this kind of tricky, deadlock-y case. Unix was not designed for in-process coroutines.

Signal handling seems like the other looming disaster for this edition of /usr/bin/luash (PowerShell meets /usr/bin/es?) but it probably seems that way to me because I'm rusty on how shells implement signal handling.

Jay