classification
Title: subprocess.Popen fails with socket._fileobject on Windows
Type: Stage:
Components: Library (Lib) Versions:
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: James.Burgess, brian.curtin, christian.heimes, htgoebel, zseil
Priority: normal Keywords: patch

Created on 2007-03-15 19:32 by htgoebel, last changed 2011-10-29 03:34 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
tst_subprocess_socket.py htgoebel, 2007-03-15 19:32
tst_subprocess_socket_client.py htgoebel, 2007-03-15 19:33
subprocess.py.patch James.Burgess, 2011-10-29 01:05
Messages (5)
msg31531 - (view) Author: Hartmut Goebel (htgoebel) Date: 2007-03-15 19:32
When using a socket._fileobject as stdin or stdout, subprocess.Popen fails:

Traceback (most recent call last):
...
  File "c:\Python25\lib\subprocess.py", line 586, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "c:\Python25\lib\subprocess.py", line 680, in _get_handles
    p2cread = msvcrt.get_osfhandle(stdin.fileno())
IOError: [Errno 9] Bad file descriptor


Enclose are a _simple_ TCPSocket-Server using subprocess.Popen for echoing some text to the socket. On Linux this works, on Windows this fails with the above traceback.

Usage:
1) In one shell start tst_subprocess_socket.py
2) In a second shell start tst_subprocess_socket_client.py
3) On Window, the server will fail and produce the obove traceback.

msg31532 - (view) Author: Hartmut Goebel (htgoebel) Date: 2007-03-15 19:33
File Added: tst_subprocess_socket_client.py
msg31533 - (view) Author: Ziga Seilnacht (zseil) * (Python committer) Date: 2007-03-16 12:31
This is a known limitation of Windows; unlike
UNIX like systems, it treats sockets and file
descriptors as completely different entities.
I don't think there is anything that Python
can do to hide this difference.
msg59368 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-06 12:51
Correct

In Windows file descriptors are handled by the MSVCR library and socket
descriptors by winsock (ws32*). For the same reason you can't use a fd
with select on Windows.

wont fix == Can't fix Windows
msg146585 - (view) Author: James Burgess (James.Burgess) Date: 2011-10-29 01:05
"Can't Fix" that is not true. 

I've just fixed this in 2.7 with a trivial change to subprocesss.py, I think it'd work in over versions too.

Note that type shenanigans are already in play in _get_handles, it's looking at the types of the parameters being passed in to decide how to get a hold of the "handle". The socket module makes a duck type of the file object. The fileno() method of the socket object returns a handle not a CRT file descriptor. This is exactly the kind of handle that _get_handles() is looking for.

So all that is needed is one more "if" to the sequence of how to get the "handle" which for a socket object would be just stdin.fileno() etc.

I've just tested this in a fairly complicated remote job queuing software (a commercial product) that has the ability to connect the spooler (on one machine) with an arbitrary server machine (linux, osx and now windows) via a socket. The job is launched with subprocess.Popen and sockets are wired into the stdin,stdout and stderr. Works beautifully now.

I've attached a patch file made with:
$ diff -c subprocess.py.ORIG subprocess.py  > subprocess.py.patch

Apply with:
$ cd  Python-2.7.1/Lib ; patch -p0 < c:/temp/subprocess.py.patch

Cheers,
- James
History
Date User Action Args
2011-10-29 03:34:47ned.deilysetnosy: + brian.curtin
2011-10-29 01:05:02James.Burgesssetfiles: + subprocess.py.patch

nosy: + James.Burgess
messages: + msg146585

keywords: + patch
2009-03-21 00:31:22ajaksu2linkissue1535504 dependencies
2008-01-06 12:51:47christian.heimessetstatus: open -> closed
resolution: wont fix
messages: + msg59368
nosy: + christian.heimes
2007-03-15 19:32:40htgoebelcreate