classification
Title: subprocess fails on GetStdHandle in interactive GUI
Type: compile error Stage:
Components: Windows Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: astrand, atrusty, bairam, bethard, codecraig, davidschein, dserodio, grante, gwtking
Priority: high Keywords:

Created on 2005-02-17 16:23 by davidschein, last changed 2013-03-04 12:59 by gwtking. This issue is now closed.

Files
File name Uploaded Description Edit
1124861.3.patch astrand, 2007-01-30 20:04 Patch which implements solution 2, with closing
Messages (19)
msg24333 - (view) Author: davids (davidschein) Date: 2005-02-17 16:23
Using the suprocess module from with IDLE or PyWindows,
it appears that calls GetStdHandle (STD_<???>_HANDLE)
returns None, which causes an error.  (All appears fine
on Linux, the standard Python command-line, and ipython.)

For example:
>>> import subprocess
>>> p = subprocess.Popen("dir", stdout=subprocess.PIPE)

Traceback (most recent call last):
  File "<pyshell#49>", line 1, in -toplevel-
    p = subprocess.Popen("dir", stdout=subprocess.PIPE)
  File "C:\Python24\lib\subprocess.py", line 545, in
__init__
    (p2cread, p2cwrite,
  File "C:\Python24\lib\subprocess.py", line 605, in
_get_handles
    p2cread = self._make_inheritable(p2cread)
  File "C:\Python24\lib\subprocess.py", line 646, in
_make_inheritable
    DUPLICATE_SAME_ACCESS)
TypeError: an integer is required

The error originates in the mswindows implementation of
_get_handles.  You need to set one of stdin, stdout, or
strerr because the first line in the method is:
if stdin == None and stdout == None and stderr == None:
...return (None, None, None, None, None, None)

I added "if not handle: return GetCurrentProcess()" to
_make_inheritable() as below and it worked.  Of course,
I really do not know what is going on, so I am letting
go now...

def _make_inheritable(self, handle):
..."""Return a duplicate of handle, which is inheritable"""
...if not handle: return GetCurrentProcess()
...return DuplicateHandle(GetCurrentProcess(), handle,
....................................GetCurrentProcess(),
0, 1,
....................................DUPLICATE_SAME_ACCESS)
msg24334 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2005-08-13 20:37
Logged In: YES 
user_id=945502

I ran into a similar problem in Ellogon (www.ellogon.org)
which interfaces with Python through tclpython
(http://jfontain.free.fr/tclpython.htm).

My current workaround is to always set all of stdin, stdout,
and stderr to subprocess.PIPE.  I never use the stderr pipe,
but at least this keeps the broken GetStdHandle calls from
happening.

Looking at the code, I kinda think the fix should be::

    if handle is None:
        return handle
    return DuplicateHandle(GetCurrentProcess(), ...

where if handle is None, it stays None.  But I'm also
probably in over my head here.
msg24335 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2005-09-26 14:53
Logged In: YES 
user_id=945502

This issue was discussed on comp.lang.python[1] and Roger
Upole suggested:

"""
Basically, gui apps like VS don't have a console, so
GetStdHandle returns 0.   _subprocess.GetStdHandle
returns None if the handle is 0, which gives the original
error.  Pywin32 just returns the 0, so the process gets
one step further but still hits the above error.

Subprocess.py should probably check the
result of GetStdHandle for None (or 0)
and throw a readable error that says something like
"No standard handle available, you must specify one"
"""

[1]http://mail.python.org/pipermail/python-list/2005-September/300744.html
msg24336 - (view) Author: craig (codecraig) Date: 2006-10-13 15:54
Logged In: YES 
user_id=1258995

On windows, this seems to work

from subprocess import *
p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)

....in some cases (depending on what command you are
executing, a command prompt window may appear).  Do not show
a window use this...

import win32con
p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE,
creationflags=win32con.CREATE_NO_WINDOW)

...google for Microsoft Process Creation Flags for more info
msg24337 - (view) Author: Peter Åstrand (astrand) * (Python committer) Date: 2007-01-22 19:36
The following bugs have been marked as duplicate of this bug:

1358527
1603907
1126208
1238747

msg24338 - (view) Author: Peter Åstrand (astrand) * (Python committer) Date: 2007-01-29 21:42
Some ideas of possible solutions for this bug:

1) As Roger Upole suggests, throw an readable error when GetStdHandle fails. This would not really change much, besides of subprocess being a little less confusing. 

2) Automatically create PIPEs for those handles that fails. The PIPE could either be left open or closed. A WriteFile in the child would get ERROR_BROKEN_PIPE, if the parent has closed it. Not as good as ERROR_INVALID_HANDLE, but pretty close. (Or should I say pretty closed? :-)

3) Try to attach the handles to a NUL device, as 1238747 suggests. 

4) Hope for the best and actually pass invalid handles in startupinfo.hStdInput, startupinfo.hStdOutput, or
startupinfo.hStdError. It would be nice if this was possible: If GetStdHandle fails in the current process, it makes sense that GetStdHandle will fail in the child as well. But, as far as I understand, it's not possible or safe to pass invalid handles in the startupinfo structure. 

Currently, I'm leaning towards solution 2), with closing the parents PIPE ends. 
msg24339 - (view) Author: Peter Åstrand (astrand) * (Python committer) Date: 2007-01-30 20:04
File Added: 1124861.3.patch
msg24340 - (view) Author: Peter Åstrand (astrand) * (Python committer) Date: 2007-01-30 20:05
Please review 1124861.3.patch. 
msg24341 - (view) Author: Peter Åstrand (astrand) * (Python committer) Date: 2007-02-06 15:43
I've applied 1124861.3.patch to both trunk (rev 53646) and the release25-maint branch (rev 53647). 
msg24342 - (view) Author: Grant Edwards (grante) Date: 2007-04-18 19:20
I _think_ this traceback from a program running under Pythonw 2.4.3 Enthought Edition on WinXP is caused by the same underlying bug (in this case, stdin was set to PIPE, and stdout/stderr were both left to default):

     File "subprocess.pyc", line 533, in __init__
     File "subprocess.pyc", line 607, in _get_handles
     File "subprocess.pyc", line 634, in _make_inheritable
   WindowsError: [Errno 6] The handle is invalid

msg59536 - (view) Author: Daniel Serodio (dserodio) Date: 2008-01-08 13:46
I found this bug via this post:
http://mail.python.org/pipermail/python-list/2007-April/436275.html

And I think 2.5.1 still has this bug. I'm not familiar with Python
bugtracker's "ettiquete", should I reopen this bug?
msg62450 - (view) Author: Andrew Trusty (atrusty) Date: 2008-02-16 05:14
I agree with Daniel, I think this bug or a variant is still present in
2.5.1 because my wxPython app on Windows XP would fail to execute a
Popen with only stdout using PIPE but succeeded with the described
workaround of having stdout, stderr, and stdin set to PIPE.
msg62457 - (view) Author: Daniel Serodio (dserodio) Date: 2008-02-16 14:35
Is there any chance of having this fixed for 2.5.2 ?
msg101416 - (view) Author: (bairam) Date: 2010-03-21 10:29
I have tested this form in Python26 ,It works
>>> import subprocess
>>> p = subprocess.Popen("cmd.exe dir", stdout=subprocess.PIPE)
>>> p.communicate()
('Microsoft Windows XP [Version 5.1.2600]\r\n(C) Copyright 1985-2001 Microsoft Corp.\r\n\r\nC:\\Python26>', None)
msg101453 - (view) Author: Daniel Serodio (dserodio) Date: 2010-03-21 20:19
Still doesn't work for me with Python 2.6.5 on Windows 7 64-bit, but apparently for a different reason.
When I type p.communicate() as in bairam's example, it blocks with no output.
msg101464 - (view) Author: (bairam) Date: 2010-03-21 21:57
shell vs subprocess
The problem is the default option "Shell" is "False" , you you write commands you should turn on the shell.
for example try this to show the content of directory " C:\Python26
":
>>>import subprocess
>>> p = subprocess.Popen("dir", shell=True,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> stdout,stderr=p.communicate()
>>> print stderr # if errors doe not occur ,it is empty!

>>> print stdout
msg101514 - (view) Author: Daniel Serodio (dserodio) Date: 2010-03-22 14:59
Ah, it works now. Thanks.
msg183441 - (view) Author: Tomgu (gwtking) Date: 2013-03-04 12:56
Here I get one problem need help. 
In python 2.7.5 version I write a script "test.py" as below
###
import subprocess
fileID=file('test1.log','w')
process_response = subprocess.call('netsh wlan show\ interface',stdout=fileID,shell=True)
fileID.flush()
fileID.close()
###
It run in windows7 os MSDOS console is OK
and I try to run it in STAF command

C:\> STAF local PROCESS START COMMAND python C:\>test.py WAIT STDERRTOSTDOUT RETURNSTDOUT

Response
--------
{
  Return Code: 1
  Key        : <None>
  Files      : [
    {
      Return Code: 0
      Data       : Traceback (most recent call last):
  File "C:\test.py", line 3, in <module>
    process_response = subprocess.call('netsh wlan show interface',stdout=fileID
,shell=True)
  File "C:\Python27\lib\subprocess.py", line 493, in call
    return Popen(*popenargs, **kwargs).wait()
  File "C:\Python27\lib\subprocess.py", line 672, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "C:\Python27\lib\subprocess.py", line 784, in _get_handles
    p2cread = self._make_inheritable(p2cread)
  File "C:\Python27\lib\subprocess.py", line 823, in _make_inheritable
    _subprocess.DUPLICATE_SAME_ACCESS)
WindowsError: [Error 6] The handle is invalid

    }
  ]

I just start to learn python, I dont know how to resolve, need help.
msg183442 - (view) Author: Tomgu (gwtking) Date: 2013-03-04 12:59
Update, the python version is '2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)]'
History
Date User Action Args
2013-03-04 12:59:46gwtkingsetmessages: + msg183442
2013-03-04 12:56:24gwtkingsetversions: + Python 2.7, - Python 2.6
nosy: + gwtking

messages: + msg183441

type: compile error
2010-03-22 14:59:14dserodiosetmessages: + msg101514
2010-03-21 21:57:20bairamsetmessages: + msg101464
versions: + Python 2.6, - Python 2.5
2010-03-21 20:19:34dserodiosetmessages: + msg101453
2010-03-21 10:29:13bairamsetnosy: + bairam
messages: + msg101416
2008-02-16 14:35:56dserodiosetmessages: + msg62457
2008-02-16 05:14:17atrustysetnosy: + atrusty
messages: + msg62450
versions: + Python 2.5, - Python 2.4
2008-01-08 13:46:27dserodiosetnosy: + dserodio
messages: + msg59536
2005-02-17 16:23:09davidscheincreate