This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Use the new selectors module in the subprocess module
Type: enhancement Stage: resolved
Components: Versions: Python 3.4
Status: closed Resolution: fixed
Dependencies: 19172 Superseder:
Assigned To: Nosy List: giampaolo.rodola, neologix, pitrou, python-dev, vstinner
Priority: normal Keywords: needs review, patch

Created on 2013-09-04 18:41 by vstinner, last changed 2022-04-11 14:57 by admin. This issue is now closed.

File name Uploaded Description Edit
subprocess_selectors.diff neologix, 2013-10-05 14:42 review
subprocess_selectors-1.diff neologix, 2013-10-30 19:39 review
subprocess_selectors-2.diff neologix, 2013-11-01 10:37 review
subprocess_selectors-3.diff neologix, 2013-11-05 19:40 review
Messages (8)
msg196936 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-09-04 18:41
Python 3.4 has a new selectors module (issue #16853). It would be nice to use it instead of select.poll or in the subprocess module.
msg196937 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-09-04 18:44
Other modules using or select.poll() for more than 1 file descriptor:

- asyncore
- multiprocessing.connection
- multiprocessing.forkserver
msg196995 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2013-09-05 13:08
It's likely that asyncore won't be able to take any practical advantage from this integration.

To say one, epoll()/kqueue() pollers won't bring any speedup over select()/poll() because of how asyncore.loop() function is implemented (see and

Also, the new selectors module only takes read and write events into account, whereas asyncore explicitly closes dispatcher in case of disconnection events (POLLOUT, etc).

In summary I'd say it's a lot wiser to leave asyncore alone and consider it frozen.
msg198989 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-10-05 14:42
Here's a patch updating subprocess to use selectors.
(It depends on the new keys() method - issue #19172.)
msg201774 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-10-30 19:39
Here's an updated patch using the new selector.get_map() method.
It removes ~100 lines to subprocess, which is always nice.
msg202234 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-11-05 19:40
Here's an updated patch with a better logic: in the previous version - based on current poll-based implementation, the FD was inferred from the event (i.e. read ready -> stdout/stderr, write ready -> stderr). The new version directly checks the ready file object instead. I also added an extra safety in case an unknown FD is returned (which should never happen).
msg202241 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-11-05 20:36
subprocess_selectors-3.diff looks good to me (this patch does not
remove any test :-))

I created the issue #19506 for the memoryview optimization.
msg202441 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-11-08 18:55
New changeset 71b618f0c8e9 by Charles-François Natali in branch 'default':
Issue #18923: Update subprocess to use the new selectors module.
Date User Action Args
2022-04-11 14:57:50adminsetgithub: 63123
2013-11-09 06:48:46neologixsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2013-11-08 18:55:45python-devsetnosy: + python-dev
messages: + msg202441
2013-11-05 20:36:33vstinnersetmessages: + msg202241
2013-11-05 19:40:37neologixsetfiles: + subprocess_selectors-3.diff
nosy: + pitrou
messages: + msg202234

2013-11-01 10:37:03neologixsetfiles: + subprocess_selectors-2.diff
2013-10-30 19:39:47neologixsetfiles: + subprocess_selectors-1.diff

messages: + msg201774
2013-10-05 14:42:42neologixsetfiles: + subprocess_selectors.diff

dependencies: + selectors: add keys() method

keywords: + patch, needs review
nosy: + neologix
messages: + msg198989
stage: patch review
2013-09-05 13:08:38giampaolo.rodolasetmessages: + msg196995
2013-09-05 12:42:56giampaolo.rodolasetnosy: + giampaolo.rodola
2013-09-04 18:44:54vstinnersetmessages: + msg196937
2013-09-04 18:41:35vstinnercreate