Title: Refactor subprocess.Popen to let a subclass handle IO asynchronously
Type: enhancement Stage:
Components: Versions: Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, martius, yselivanov
Priority: normal Keywords: patch

Created on 2016-09-27 14:26 by martius, last changed 2017-07-14 16:09 by giampaolo.rodola.

File name Uploaded Description Edit
popen_execute_child_refactoring.patch martius, 2016-09-27 14:26 popen_execut_child_refactoring.patch, should work with python 3.4+ review
Messages (2)
msg277517 - (view) Author: Martin Richard (martius) * Date: 2016-09-27 14:26

Currently, subprocess.Popen performs blocking IO in its constructor (at least on Unix): it reads on a pipe in order to detect outcome of the pre-exec and exec phase in the new child. There is no way yet to modify this behavior as this blocking call is part of a long Popen._execute_child() method.

This is a problem in asyncio (asyncio.subprocess_exec and asyncio.subprocess_shell).

I would like to submit a patch which breaks Popen.__init__() and Popen._execute_child() in several methods so it becomes possible to avoid blocking calls (read on pipe and waitpid) by overriding a few private methods without duplicating too much code. The goal is to use it in asyncio, as described in this pull request (which currently monkey-patches Popen):

This patch only targets the unix implementation.

Thanks for your feedback.
msg298358 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2017-07-14 16:09
Is the main goal to just make Popen.__init__ non-blocking? If not, #1191964 aims at providing non-blocking reads/writes (including on Windows).
Date User Action Args
2017-07-14 16:09:07giampaolo.rodolasetnosy: + giampaolo.rodola
messages: + msg298358
2016-09-30 18:44:38terry.reedysetversions: + Python 3.7, - Python 3.6
2016-09-30 18:43:59terry.reedysetnosy: + yselivanov
2016-09-27 14:26:43martiuscreate