Re: Redirecting stdin/stdout/stderr in embedded environment

[Paul Moore <pf_moore@...>]

Mark Edgar <medgar123 <at> gmail.com> writes:

> Does your application change these standard handles just once, at its
> startup and before it initializes Lua?

No, I'm afraid not. It changes them dynamically. It's not actually my 
application as such - I'm writing a plugin for 4NT (cmd.exe replacement from 
JP Software) which embeds a Lua interpreter into the command line.

> If so, one solution would be
> for your application to modify the standard handles, and then "exec"
> itself.  Then, stdin/out/err would be setup as CreateProcess says.

As I said, that doesn't work.

> If you can't do this, then you can use _fdopen and make use of the
> fact that MSVCRT seems to reuse standard FILE slots:
> 
> fclose(stdin);
> HANDLE h = CreateFooHandle(yadda, yadda, yadda);
> FILE *f = _fdopen(_open_osfhandle((long)h, _RD_ONLY), "rb");
> assert( f == stdin );  /* hopefully! */
> /* rinse, repeat for stdout, stderr */
> lua_open()
> 
> Yes, this is a dirty, ugly hack.  Please clearly mark it as such in
> your comments!

Actually, I tried something very similar (using _dup2 to avoid the "hopefully" 
aspect of the above :-)) The problem is that when stdin gets reopened, the 
original stdin is closed. That causes 4NT to shut down when the Lua call 
returns. I suspect what's happening is that when I use input redirection, 4NT 
temporarily resets STD_INPUT_HANDLE, but retains the old value for when the 
command completes. If my command closes the original STD_INPUT_HANDLE (as a 
result of reopening stdin), 4NT sees its standard input showing EOF, assumes 
it's finished, and shuts down.

So I really do need to point Lua's io.stdin/out/err at different FILE* 
handles, without affecting the C stdin/out/err handles at all.

Maybe the simplest way would be a small tweak to liolib.c to expose a C API 
for resetting the FILE* handles that io.stdin/out/err point to. I can do this 
locally, but would such a change be considered for inclusion in the next 
release of Lua?

Thanks,
Paul.