classification
Title: subprocess Popen objects are not thread safe w.r.t. wait() and returncode being set
Type: behavior Stage: commit review
Components: Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: gregory.p.smith, python-dev
Priority: normal Keywords: patch

Created on 2014-04-17 21:49 by gregory.p.smith, last changed 2014-04-23 08:35 by gregory.p.smith. This issue is now closed.

Files
File name Uploaded Description Edit
subprocess_wait_problem.py gregory.p.smith, 2014-04-17 21:49
issue21291-fix-gps01.diff gregory.p.smith, 2014-04-18 01:35 review
issue21291-patch-with-test-gps01.diff gregory.p.smith, 2014-04-23 06:36 review
Messages (4)
msg216757 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2014-04-17 21:49
Executing the supplied test code you either get:

sys.version = 3.4.0+ (3.4:635817da596d, Apr 17 2014, 14:30:34) 
[GCC 4.6.3]
--------
Results with <class 'subprocess.Popen'>:
  r0    = None, expected None 
  r1    = None, expected None 
  ri0   = None, expected None 
  ri1   = -9, expected -9 
  ri2   = -9, expected -9 
  r2    = 0, expected -9 *** MISMATCH ***
  r3    = 0, expected -9 *** MISMATCH ***

or

Results with <class 'subprocess.Popen'>:
  r0    = None, expected None 
  r1    = None, expected None 
  ri0   = None, expected None 
  ri1   = -9, expected -9 
  ri2   = -9, expected -9 
  r2    = 0, expected -9 *** MISMATCH ***
  r3    = 0, expected -9 *** MISMATCH ***

At first glance it appears that the .returncode attribute is not safely set after the wait call... This test code is using the Popen object from multiple threads at once.
msg216770 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2014-04-18 01:35
Attaching a proposed fix for this issue.  It should make the wait() and poll() methods thread safe.  I need to turn the reproducer code into an actual test case and add more test cases for coverage of all code paths being touched.

I haven't examined the windows side of this code for the issue, it may also be useful there.
msg217057 - (view) Author: Roundup Robot (python-dev) Date: 2014-04-23 07:38
New changeset 5d745d97b7da by Gregory P. Smith in branch '3.4':
subprocess's Popen.wait() is now thread safe so that multiple threads
http://hg.python.org/cpython/rev/5d745d97b7da

New changeset df45d0336dad by Gregory P. Smith in branch 'default':
subprocess's Popen.wait() is now thread safe so that multiple threads
http://hg.python.org/cpython/rev/df45d0336dad
msg217059 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2014-04-23 08:35
This fix is also present in subprocess32 3.2.6 on PyPI for use on Python 2.
History
Date User Action Args
2014-04-23 08:35:30gregory.p.smithsetstatus: open -> closed
type: behavior
messages: + msg217059

resolution: fixed
stage: commit review
2014-04-23 07:38:32python-devsetnosy: + python-dev
messages: + msg217057
2014-04-23 06:37:02gregory.p.smithsetfiles: + issue21291-patch-with-test-gps01.diff
2014-04-18 01:35:25gregory.p.smithsetfiles: + issue21291-fix-gps01.diff
keywords: + patch
messages: + msg216770
2014-04-17 21:49:43gregory.p.smithcreate