This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Receiving messages from subprocess does not work on Mac OS X
Type: behavior Stage: resolved
Components: macOS Versions: Python 3.5, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: berker.peksag, ned.deily, pyguy, ronaldoussoren
Priority: normal Keywords:

Created on 2017-02-21 00:46 by pyguy, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
testwrapper.py pyguy, 2017-02-21 00:46 The simple test program needed to reproduce the problem
Messages (7)
msg288250 - (view) Author: (pyguy) Date: 2017-02-21 00:46
When I use the subprocess module to run a another process, I expect to be able to read the STDOUT and STDERR of the other process. This is not possible on Mac OS X. The attached test program has been tested on Linux and Windows and does work as expected. It pauses at the process.stdout.readline() line of code every time on Mac OS X.
msg288251 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-02-21 00:55
Sorry, your program works for me on macOS 10.12.3 with the current python.org 2.7.13, the Apple-supplied system Python 2.7.10, and, with adding a decode to the readline, with 3.6.0.
msg288252 - (view) Author: (pyguy) Date: 2017-02-21 01:05
The program failed for me on Mac OS 10.4.11 using Python 2.7.12 and Mac OS 10.6.8 using Python 2.7.13.
msg288254 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-02-21 02:06
There certainly could be differences in behavior considering how old 10.6.8 and 10.4.11 are.  I'm not a subprocess expert but it seems to me that, if your program hangs doing a readline from process.stdout, chances are you are running into a pipe buffer deadlock as warned about in the subprocess module documentation:

"Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process."

https://docs.python.org/dev/library/subprocess.html#popen-objects
msg288257 - (view) Author: (pyguy) Date: 2017-02-21 03:20
Using communicate fixed the problem. Here is the program that works for me on Mac OS 10.4.11 with Python 2.7.12:

import subprocess
import time

print("Launch started")
program_name = "top"
list = [program_name]
process = subprocess.Popen(list)
while process.poll() == None:
    print("subprocess still running")
    print("Value = " + process.communicate())
    time.sleep(0.1)
    
print("Exit")



Thank you Ned.
msg288263 - (view) Author: (pyguy) Date: 2017-02-21 04:10
The program does not work the way I wanted it to. I want the output of the top command to be seen only by the python program. The program I made causes the top command to print its output to the terminal. Using subprocess.Popen() with stdout=subprocess.PIPE does prevent the top command's output from being printed to the terminal. Now I need a way to actually work with that output. Would you know a way to do this?

I can't use the communicate() function because it blocks until the top command quits.
msg288275 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-02-21 07:33
Please use Stack Overflow or python-list for usage questions (and please do not reopen this issue again)
History
Date User Action Args
2022-04-11 14:58:43adminsetgithub: 73795
2017-02-21 07:33:48berker.peksagsetstatus: open -> closed

nosy: + berker.peksag
messages: + msg288275

resolution: works for me -> not a bug
2017-02-21 04:10:48pyguysetstatus: closed -> open
resolution: not a bug -> works for me
messages: + msg288263
2017-02-21 04:03:54ned.deilysetstatus: open -> closed
resolution: fixed -> not a bug
stage: resolved
2017-02-21 03:20:14pyguysetresolution: works for me -> fixed
messages: + msg288257
2017-02-21 02:06:51ned.deilysetmessages: + msg288254
2017-02-21 01:05:20pyguysetstatus: pending -> open

messages: + msg288252
2017-02-21 00:55:35ned.deilysetstatus: open -> pending
resolution: works for me
messages: + msg288251
2017-02-21 00:46:41pyguycreate