This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Not Inheriting File Descriptors on Windows?
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.1
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: 3vi1, amaury.forgeotdarc, brian.curtin, sbt
Priority: normal Keywords:

Created on 2011-06-03 20:35 by 3vi1, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (6)
msg137582 - (view) Author: Jesse Litton (3vi1) Date: 2011-06-03 20:35
On Windows, I've been trying to call a test script that gets its I/O handled via file descriptors 3 & 4 

  socat EXEC:"python test.py userid",pty,fdin=3,fdout=4 TCP4:server:23,crlf

But I'm getting "[Errno 9] Bad file descriptor" when the python script attempts to os.fdopen(3, 'r').

I've tried just piping an echo's output redirected (3<&1 I think... though I might have that backwards as I've tried it every conceivable way) into the script, but I always get the bad file descriptor error.

I can create a pipe() in the program I can get an actual FD 3 & 4... but they seem to have no relation to the FD's that were set up by the invoking command-line/script.

I'm new to Python - is there something simple I'm overlooking, or is this a known bug that I just haven't been able to find in my last few hours of web searches?  I can't believe I would be the only one doing this type of redirection on Windows.

Thanks in advance for any guidance/resolution you can offer.
msg137583 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2011-06-03 20:37
Can you provide a simple test script?
msg137586 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2011-06-03 20:56
Is "socat" a cygwin utility?  In this case, you should use the python interpreter built for the cygwin platform.

I'm quite certain that the standard win32 python cannot work the way you want.  Keep in mind that on Windows, file descriptors are emulated; only HANDLEs are really inherited between processes.  And I know that the Microsoft C run-time only initializes file descriptors 0, 1 and 2 from the three standard handles.

The cygwin compatibility layer certainly does it differently and uses other ways to pass file descriptors between processes.
msg137660 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2011-06-04 19:31
Although Windows fds are not inheritable, the handles associated with fds can be made inheritable.

A workaround for the fact fds are not inheritable is the following pattern:

1) The parent process converts the fd to a handle using _get_osfhandle(fd).

2) The parent process makes the handle inheritable (if it is not already) using SetHandleInformation() or DuplicateHandle().

3) The parent process starts the child process and passes the handle number on the commandline.

4) The child process gets the handle number by parsing the commandline.

5) The child process uses _open_osfhandle(handle, flags) to convert the handle into an fd.  

Note that Python has open_osfhandle() and get_osfhandle() in the msvcrt module.  In C, _open_osfhandle() and _get_osfhandle() are declared in io.h.
msg137684 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2011-06-05 07:00
Yes, that's probably how the cygwin runtime library works. But this method is difficult to use from a shell script.
msg137827 - (view) Author: Jesse Litton (3vi1) Date: 2011-06-07 13:51
Yes, socat was compiled under cygwin.  I'll have to investigate whether I want to go the route of cygwin separately or just just leave the script as a pure stdin/out filter (since neither I nor the users have the cygwin environment installed).

After reading what you said, I re-read the MS notes (http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/redirection.mspx?mfr=true) and finally noticed the 'specific to each tool' text next to the description for handles 3-9.
 
Just to be sure, I came up with this simple test:

   echo test 1>&3 | set /P TEST= 0<&3

Which basically squashes my misconception, as it doesn't work.

So, I believe now that what I wanted is beyond the capabilities of the common Windows command shell, and this can be closed/invalid as not being a Python issue.  I also wish I was home using my Linux box about now.  :)

Thank you all very much for taking the time to help educate me on this!
History
Date User Action Args
2022-04-11 14:57:18adminsetgithub: 56471
2011-06-07 13:51:343vi1setstatus: open -> closed
resolution: not a bug
messages: + msg137827
2011-06-05 07:00:51amaury.forgeotdarcsetmessages: + msg137684
2011-06-04 19:31:02sbtsetnosy: + sbt
messages: + msg137660
2011-06-03 20:56:56amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg137586
2011-06-03 20:37:01brian.curtinsetnosy: + brian.curtin
messages: + msg137583
2011-06-03 20:35:483vi1create