New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
subprocess failing in GUI applications on Windows #48155
Comments
I'm getting a 'The handle is invalid' error when using subprocess.Popen The easiest way to reproduce this is using run PythonWin from a Windows import subprocess
p = subprocess.Popen(["python", "-c", "print 32"], stdin=None,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
File "C:\Python26\lib\subprocess.py", line 588, in init_
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "C:\Python26\lib\subprocess.py", line 703, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "C:\Python26\lib\subprocess.py", line 748, in _make_inheritable
DUPLICATE_SAME_ACCESS)
WindowsError: [Error 6] The handle is invalid Specifying PIPE for all subprocess.Popen handles is the only way I've p = subprocess.Popen(["python", "-c", "print 32"],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p.stdin.close()
p.communicate()
('32\r\n', '') |
This seems to be somewhat related to bpo-1124861: |
The failure is in the DuplicateHandle call that subprocess makes on, in Two cases here:
[http://msdn.microsoft.com/en-us/library/ms683231(VS.85).aspx]
In this case you don't get the error that Todd described, because the
The code path in subprocess.py then calls My *guess* at what the problem is stems from this comment in the MSDN [http://msdn.microsoft.com/en-us/library/ms682075(VS.85).aspx]
My guess is that the stdin handle (3) is inherited from the shell Anyone else have more light to shed on this? If this is the case I'm not sure what a possible or good solution could |
I do not really know if this is related to this, but I suspect yes. Subprocess Startup Error It is certainly neither a firewall issue, nor a tkinter issue (tkinter No problem with the 3.0b2 realease, 3.0b3: not tested. |
jmfauth: please try to run idle from a command prompt:
Do you see interesting output there? |
Microsoft Windows 2000 [Version 5.00.2195] C:\>cd python30 C:\Python30>python Lib/idlelib/idle.py
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python30\lib\idlelib\run.py", line 76, in main
sockthread.set_daemon(True)
AttributeError: 'Thread' object has no attribute 'set_daemon' C:\Python30> Sorry if I can help here, things like socket and subprocess are not my |
Benjamin, I think you're responsible. |
There are also instances of set_daemon left in socketserver and |
The idle problem has already been fixed, and I got the socket server one |
Just for information and from an end user perspective. I have tried to replace the socketserver.py from the original 3.0rc1 Script difference (line 568): if self.daemon_threads:
t.daemon = True
t.start() and if self.daemon_threads:
t.set_daemon(True)
t.start() Unfortunately, no luck, I'm still getting exactly the same error AttributeError: 'Thread' object has no attribute 'set_daemon' |
Jean-Michel, you'll need to apply the idlelib/run.py patch; replace sockthread.set_daemon(True) with sockthread.daemon = True around line 75. |
Uh, the Idle bug reported by Jean-Michel is a completely different bug Please re-open this bug, as the subprocess issue I have reported is |
Thanks, patch applied on w2k+sp4 box, swiss french, Python 3.0rc1. |
Georg Brandl thank you, i was having the same problem, but now is fixed |
I just re-tested this in cpython trunk at changeset and the issue does not appear to be reproducible. I first launched IDLE by running "python lib\idlelib\idle.py". Then I entered the following: Python 3.3a0 (default, Apr 2 2011, 21:55:40) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import sys
>>> sys.getwindowsversion()
sys.getwindowsversion(major=5, minor=1, build=2600, platform=2, service_pack='Service Pack 3')
>>> import subprocess
>>> PIPE = subprocess.PIPE
>>> p = subprocess.Popen(["python", "-c", "print(32)"], stdin=None, stdout=PIPE, stderr=PIPE)
>>> p
<subprocess.Popen object at 0x00F85610>
>>> p.returncode
>>> Popen() did not raise any exceptions. I also tried in Python 2.7 and the Popen called succeeded there as well. So my inability to reproduce the issue does not necessarily mean that the issue is fixed, but rather that it is likely dependent on the version of Windows on which Python is running. That being said, the linked issue bpo-1124861 logs what appears to be the same issue and it was fixed in Python 2.6. So maybe I'm just not old enough to have encountered this issue :) Anyways, this issue probably can be closed as either a duplicate or fixed incidentally. |
I still see this problem (in Python 2.7.1 and Python 3.1.2). When testing using idle, you'll need to launch using "pythonw.exe", i.e.: pythonw.exe lib\idlelib\idle.py |
Ahh okay. I've reproduced it in trunk at changeset 053bc5ca199b. As suggested, I ran: PCBuild\pythonw.exe lib\idlelib\idle.py Python 3.3a0 (default, Apr 2 2011, 21:55:40) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import subprocess
>>> subprocess.Popen(["python", "-c", "print(32)"], stdin=None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
subprocess.Popen(["python", "-c", "print(32)"], stdin=None, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "C:\dev\cpython\cpython\lib\subprocess.py", line 732, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "C:\dev\cpython\cpython\lib\subprocess.py", line 903, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "C:\dev\cpython\cpython\lib\subprocess.py", line 946, in _make_inheritable
_subprocess.DUPLICATE_SAME_ACCESS)
WindowsError: [Error 6] The handle is invalid
>>> So, the issue is definitely not fixed. |
In 3.2.2, starting IDLE from a Command Prompt, I get the same error as Denver. Starting IDLE 'normally', from the icon, there is no error. So even though Command Prompt detaches pythonw in some way and gives me a new prompt to do something else, there is something different about the pythonw process that trips up subprocess. Changing ["python", "-c", "print(32)"] to ["notepad"] gives the same result. Running in the interactive interpreted eliminated the error. So does changing stdin=None to stdin=subprocess.PIPE eliminates the error and brings up Notepad, which seems to operate normally. (I am guessing that it ignores stdin, etc.). I see that that the latter was known in the original post and plausibly diagnosed by Trent Mick, but never fixed because it was not obvious how to do so. Can the WindowsError be caught in subproccess, and if it is this error (6), follow the alternate patch that is used for NULL returns? Could that cause worse problems that the current situation? If this is unfixable due to Windows behavior, perhaps we should say something in the doc. |
I cannot reproduce this problem with 3.4.1 or 3.5.a0 on Windows 7. |
Mark, you did not specify which of the variations you tried ;-) I reproduce as I said in my last message. C:\Programs\Python34>pythonw -m idlelib Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [
>>> import subprocess
>>> p = subprocess.Popen(["python", "-c", "print(32)"], stdin=None,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Traceback (most recent call last):
...
_winapi.DUPLICATE_SAME_ACCESS)
OSError: [WinError 6] The handle is invalid
Change startup command from pythonw to python, no error, and it runs.
>>> p.communicate()
(b'32\r\n', b'') |
*facepalm* looks like I've run it with the wrong exe, sorry about the noise :-( |
Probably the same issue: everything works when called from command line, but when called via a desktop shortcut I get Changing pythonw to python solved the problem. ...this could be expected behavior. If anyone is interested I can attach my test files. |
I reran test with IDLE started 4 ways: 3.8.0a2+ 32 bit debug python, built today, opened from python icon, then idle started with 'import idlelib.idle' # and it ran correctly all 4 times.
>>> import subprocess
>>> p = subprocess.Popen(["python", "-c", "print(32)"], stdin=None,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> p.communicate()
(b'32\r\n', b'') Anyone with the same or similar failure on current Python (now 3.7, 3.8) should open a new issue and give careful details -- release or build, system, startup procedure, and a reproducing example only using stdlib modules (no PythonWin). |
This is a problem only in Windows 7, which we should have addressed years ago. It's common enough that we shouldn't leave it as just an unresolved third-party issue. I discussed the problem and suggested a workaround in bpo-25492, which I closed as a duplicate of this issue four years ago. |
Since Win 10 was released a year after Jun 2014, I must have still been on Win 7 when I saw the failure. I believe there have some patches to filenos and subprocess since, but I don't know the relation between filenos and file handles. A current test on a Win 7 machine similar to what I did would still be good. Perhaps it would have been better to reopen bpo-25492, but I am adding the current Windows people nosy here. |
Python 3.8 is the last version to support Windows 7 (i.e. NT 6.1). It should get bug fixes through Spring 2021, so we have a couple more years to fix this. We should constrain the fix to just older versions of Windows (prior to 6.2) and the classic console file handles (i.e. the lower 2 bits are set such as 3, 7, 11) that they use. Otherwise a bad standard handle should fail the call. |
For whatever the reason and Windows version, it's still the case that _get_handles() should work around any bad standard handles in the current process. In bpo-25492, I suggested checking os.get_handle_inheritable(). That's too permissive. It should require a valid file handle, checked via _winapi.GetFileType(). For example: if stdin is None:
p2cread = _winapi.GetStdHandle(_winapi.STD_INPUT_HANDLE)
if p2cread is not None:
try:
_winapi.GetFileType(p2cread)
except OSError:
p2cread = None |
Python 3.8 and 3.9 are now in security-only mode. Suggesting to close this. |
Closing per Erlend, it's been several days with no comments to the contrary. A |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: