classification
Title: subprocess.Popen.communicate(...) hangs on Windows
Type: behavior Stage: resolved
Components: IO, Library (Lib), Windows Versions: Python 3.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: tim.golden Nosy List: Alex Quinn, BreamoreBoy, Saimadhav.Heblikar, akira, paroga, steve.dower, terry.reedy, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2010-05-05 21:45 by Alex Quinn, last changed 2017-06-18 19:49 by tim.golden. This issue is now closed.

Messages (10)
msg105088 - (view) Author: Alex Quinn (Alex Quinn) Date: 2010-05-05 21:45
After using subprocess.Popen(...).communicate(), the session hangs.

c:\>python31
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import Popen, PIPE
>>> Popen(["echo","Hello!"], stdout=PIPE).communicate()[0]
b'Hello!\n'
>>>

At this point, the session stops responding to keyboard input.



=========================================
MORE DETAILS:

Neither Ctrl-C, Ctrl-Z, nor Ctrl-D will break out of it, but Ctrl-Break does break out of it.

The task manager shows no CPU activity for the process.

I've seen problem with Python v3.1.2 and v2.6.1 on Windows 7.

It works fine under Linux:

$ python3.1
Python 3.1 (r31:73572, Mar 22 2010, 14:57:00)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import Popen, PIPE
>>> Popen(["echo","Hello!"], stdout=PIPE).communicate()[0]
b'Hello!\n'
>>> print("No problems under Linux")
No problems under Linux
>>>

Is this bug type "crash" or "behavior"?  I went with the former, since it ends the usable Python session.
msg113257 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-08-08 13:04
I can't reproduce this using Windows Vista.
msg113283 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2010-08-08 16:56
I can't reproduce on W7. Strangely, though, although my banner suggests that I have exactly the same build as you, I get a "\r\n" at the end of the communicate bytestream, not a simple "\n" as you're getting. Do you have any environment variables or Python command-line switches which might be affecting anything?

Could you try with python31/26 -E -S just in case, please?
msg113419 - (view) Author: Alex Quinn (Alex Quinn) Date: 2010-08-09 15:05
I am on Windows 7.  I realized the "echo" command I'm piping to belongs to Cygwin.  I'll try to make a different example to either support this, or otherwise close the bug.

Thanks!
msg122087 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-11-22 04:12
I'm with Tim and Mark - can't reproduce this, so I'm closing the report. If you are able to find another case which can reproduce this, feel free to re-open.
msg132804 - (view) Author: Patrick Gansterer (paroga) Date: 2011-04-02 16:09
I can reproduce this on WinXP with 2.5, 2.7 and 3.2. On my other box (Win7 64bit) the same code works without problems.
This problem happens only when I make a subprocess.call() to executables from my msysgit installation (e.g. echo, git). A call to 'notepad' or 'svn' works without problems.

After entering the following python does not respond to any input:
>>> import subprocess
>>> subprocess.call('echo')
msg222053 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-07-01 18:32
There are certainly some problems with subprocess on Windows.
Note .../python34> pip pyflakes  installs in a second or two.
...> pyflakes -h  # Windows Command Propmpt WCP
>>> import subprocess as s; s.check_output("pyflakes -h")
console interpreter CI or Idle, all produce help output

WCP> pyflakes c:\programs\python34\lib\turtle.py  # or / instead of \
c:\programs\python34\lib\turtle.py:572: local variable 'rgb' is assigned to but never used
... 50+ lines (all < 1/2 second)
c:\programs\python34\lib\turtle.py:4139: undefined name 'exitonclick'

CI>>> s.check_output("pyflakes c:\programs\python34\lib\turtle.py")
pause, so almost think is hanging, then

c:\programs\python34\lib\lib2to3\tests\data\bom.py:2:17: invalid syntax print "BOM BOOM!"
...100 more error for various files.
pyflakes ignores file name given and checks entire stdlib.

ID>>> s.check_output("pyflakes c:\programs\python34\lib\turtle.py")
hangs indefinitely, no output

CI or ID >>> s.check_output("pyflakes c:/programs/python34/lib/turtle.py")  # / instead of \ which was ok in WCP
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "F:\Python\dev\5\py35\lib\subprocess.py", line 627, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command 'pyflakes c:/programs/python34/lib/turtle
.py' returned non-zero exit status 1

Above is CI 3.5 and Idle 3.4. subprocess and pyflakes work fine on linux. We are trying to add a feature to Idle to do above on file in editor, and are stuck here. The actual provisional code is
+            proc = Popen(args, stdout=PIPE, stderr=PIPE)
+            proc.wait()
+            output, error = map(lambda b:b.decode('utf-8'), proc.communicate())
where args is  list, which shows the same hang forever behavior.
msg222136 - (view) Author: Akira Li (akira) * Date: 2014-07-02 22:09
> ID>>> s.check_output("pyflakes c:\programs\python34\lib\turtle.py")
> hangs indefinitely, no output

It might be unrelated to the issue but "\t" is a tab; a raw-string literal should be used instead:

  >>> from subprocess import check_output
  >>> check_output(r"pyflakes c:\programs\python34\lib\turtle.py")
msg222145 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-07-02 23:05
That has been pointed out to me. I need to repost with the results after correcting the commands (there is still a problem).
msg296291 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2017-06-18 19:49
I can't reproduce either on 2.7 or on 3.5 with any of the examples shown. Closing again as not-a-bug.
History
Date User Action Args
2017-06-18 19:49:54tim.goldensetstatus: open -> closed

resolution: not a bug
messages: + msg296291
versions: - Python 2.7, Python 3.4
2014-07-02 23:05:33terry.reedysetmessages: + msg222145
2014-07-02 22:50:48brian.curtinsetnosy: - brian.curtin
2014-07-02 22:09:33akirasetnosy: + akira
messages: + msg222136
2014-07-01 18:32:31terry.reedysetnosy: + Saimadhav.Heblikar
2014-07-01 18:32:12terry.reedysetstatus: closed -> open
versions: + Python 3.4, Python 3.5, - Python 2.6, Python 2.5, Python 3.1, Python 3.2
nosy: + terry.reedy, zach.ware, steve.dower

messages: + msg222053

resolution: rejected -> (no value)
2011-04-02 16:09:02parogasetnosy: + paroga

messages: + msg132804
versions: + Python 2.5, Python 2.7, Python 3.2
2010-11-22 04:12:31brian.curtinsetstatus: open -> closed
type: crash -> behavior
messages: + msg122087

resolution: rejected
stage: resolved
2010-08-09 15:05:09Alex Quinnsetmessages: + msg113419
2010-08-08 16:56:32tim.goldensetassignee: tim.golden
messages: + msg113283
2010-08-08 13:04:54BreamoreBoysetnosy: + tim.golden, brian.curtin, BreamoreBoy
messages: + msg113257
2010-05-05 21:45:04Alex Quinncreate