Re: io.popen read and write, again

["Gary V. Vaughan" <gary@...>]

Salut Petite,


On Feb 26, 2014, at 9:14 AM, Petite Abeille <petite.abeille@gmail.com> wrote:

> Hello,
> 
> This ground was covered, for example, by Edgar Toernig, back in 2007:
> 
> http://lua-users.org/lists/lua-l/2007-10/msg00189.html
> 
> Nonetheless I would like to sanity check my current concoction :)
> 
> Basically I would like to somewhat emulate the following construct, but in Lua:
> 
> [[snip]]
> 
> lua - <<\EOF
> local aCommand = 'sqlite3 -bail | cat'
> local aHandle = assert( io.popen( aCommand, 'w' ) )
> 
> assert( aHandle:write( '.head on', '\n' ) )
> assert( aHandle:write( 'select count( * ) as count from sqlite_mister;', '\n' ) ) -- invalid table name
> print( aHandle:close() )
> EOF
> 
>> Error: near line 2: no such table: sqlite_mister
>> true	exit	0
> 
> What would be a reasonable way to capture the exit code of the initial command (sqlite3) as opposed to the piped one (cat)?
> 
> Is using PIPESTATUS reasonable enough?

That presumes /bin/sh is bash, which is not the case except on GNU/Linux -- and not all the time even there.

> Thoughts? Suggestions?
> 
> Thanks in advance.

The problem with io.popen is that you cannot access both the input and output of the pipeline from Lua.  But, since it uses the shell by default, you can work around this limitation with temporary files and shell redirections. For example (untested!):

  local fin, fout, ferr, fstat = os.tmpname (), os.tmpname (), os.tmpname (), os.tmpname ()

  local h = io.open (fin, "w")
  h:write "some SQL for pipeline stdin"
  h:close ()

  local cmd  = "{ sqlite3 -bail; printf '%s' $? > " .. fstat .. "; } | { cat; printf '%s' $? > " .. fstat .. "; }"
  local pipe = io.popen ("while " .. cmd .. "; do break; end < " .. fin .. " > " .. fout .. " 2> " .. ferr)

  local hout, herr, hstat = io.open (fout), io.open (ferr), io.open (fstat)
  local sout, serr, sstat = hout:read "*a", herr:read "*a", stat:read "*a"
  local ok = hout:close () and herr:close () and hstat:close ()

Now you have the stringified stdout and  stderr of the pipeline in sout and serr respectively, and if sstat contains any non-zeros, then one of the pipeline processes exited abnormally.

If you like, you can write stdin for the pipeline to `pipe` or read stdout from `pipe` and save a temporary file (but not both); or you could change printf's first argument to 'name: %s\n' and then map each line in fstat back to a particular pipeline process if you need to figure out what part of the pipeline bombed out.

My Specl project (http://gvvaughan.github.io/specl) has a prototype of this in lib/specl/shell.lua which I'm refactoring at the moment if you want to see a fairly general implementation.

Interestingly, the first version of this used luaposix to hook up the pipes, which required a lot more code and was considerable more hairy.  But most of the overhead is in setting up the pipes, so for no noticeable speedup in execution and swapping a dependency on /bin/sh for a dependency on luaposix, I got to maintain all the low-level grungy pipe hookups instead of letting /bin/sh do the work.

Caveat: obviously none of this works on Windows which is not POSIX compliant, can't fork processes, and doesn't have a shell (as such).

Cheers,
-- 
Gary V. Vaughan (gary AT vaughan DOT pe)

Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail