Re: io.popen and binary read mode on Windows

[Tom Sutcliffe <tomsci@...>]

> On 30 Nov 2020, at 1:40 pm, Scott Morgan <blumf@blueyonder.co.uk> wrote:
> 
> On 30/11/2020 12:59, Roberto Ierusalimschy wrote:
>>>> io.popen('lua.exe -e"print(string.char(49,26,50))"',
>>> "rb"):read"*a":byte(1,-1)
>>> stdin:1: bad argument #2 to 'popen' (invalid mode)
>>> [...]
>>> 
>>> Why does Lua 5.4 not allow me to read verbatim output of an external
>>> Windows process?
>> 
>> According to the official documentation [1]:
> ...
>> Note that the string "rb" is not in the list. :-)
>> 
>> More seriously, we really don't know the meaning of the above
>> documentation, and therefore we don't know what to implement.
>> (Maybe they mean "[rw][bt]?", but that is just speculation.)
> ...
>> [1] https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/popen-wpopen?view=msvc-160
> 
> The linked page gives an example use with "rb". Seems pretty obvious
> that you'd need to combine the text/binary modifier with the read/write
> flags, just like (the better documented) fopen.


I had to replace the default implementation of io.open with an implementation using _wfopen_s to ensure I could open files with non-ascii unicode characters in their path, because Windows makes such a mess of the narrow versions of the Win32 APIs (they aren't, and cannot be made, UTF-8 because that would be useful). So I had to dig into this quite deeply a while back. From what I can remember, "t" is implied if "b" is not present, so "r"=="rt", "w"=="wt" etc. I forget if it's POSIX that says that text mode should be default but man it's a terrible choice for a default.

But short story is yes "rb" and "wb" are valid modes and probably the only modes you should ever use on Windows (unless you actually want magic line ending translation, which generally speaking is better done at a way higher level).

> Worryingly, it also mentions:
> 
>> If used in a Windows program, the _popen function returns an invalid
>> file pointer that causes the program to stop responding indefinitely.
>> _popen works properly in a console application.
> 
> Which looks like an unpleasant catch for people embedding Lua in a
> Windows app. The linked reference[1] looks ugly, like most win32 code.

I haven't hit that particular horror, thankfully, but I don't expose io.popen and have a CreateProcess-based reimplementation instead. Because of various shortcomings with trying to make popen work with other Windows processes. Windows's POSIX APIs barely work, and to get decent behaviour I've found you generally have to ignore them and switch to either the wide versions or the Win32 equivalent.=s.

> As usual MS makes life far harder than it needs to be.

It doesn't help that the backend for docs.microsoft.com has gone through multiple iterations meaning a lot of articles have utterly broken markup, which makes it extra difficult to figure out what on earth the docs are telling you. For example the current docs for _wfopen_s say that adding "n" to the mode is how you prevent the handle being inherited by child processes; that is incorrect and it is actually "N" (at least, that is what my implementation uses based on my reading of those docs a few years ago before the most recent round of breakages).

There are a bazillion other mode flags you can set on Windows file handles (I 100% concur with "MS makes life far harder than it needs to be"!), I ended up mapping io.open(name, "r") to _wfopen_s(widename, "rbN") and io.open(name, "w") to _wfopen_s(widename, "wbN") and not allowing any other combinations, which worked for the embedded code I was working with although I admit is not necessarily going to be the desired behaviour in all applications.

Cheers,

Tom