classification
Title: test_multiprocessing_spawn: WithProcessesTestProcess.test_many_processes() leaks 5 handles on Windows
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: davin, miss-islington, pitrou, vstinner
Priority: normal Keywords: patch

Created on 2018-06-21 11:02 by vstinner, last changed 2018-06-27 13:19 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 7921 merged vstinner, 2018-06-26 00:10
PR 7960 merged miss-islington, 2018-06-27 09:41
PR 7961 closed vstinner, 2018-06-27 09:46
PR 7962 merged vstinner, 2018-06-27 09:59
PR 7963 merged vstinner, 2018-06-27 10:08
Messages (9)
msg320160 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-21 11:02
The following test leaks 5 open handles when checking for leaks using my PR 7827:

test.test_multiprocessing_spawn.WithProcessesTestProcess.test_many_processes

Command to reproduce the leak (using my PR 7827):

python -m test -R 3:3 test_multiprocessing_spawn -m test.test_multiprocessing_spawn.WithProcessesTestProcess.test_many_processes
msg320457 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-26 00:11
It seems like Popen of multiprocess.popen_spawn_win32 has a race condition. If the child process is killed in the parent using terminate() before the child process "steals" rhandle, rhandle remains open. I wrote the PR 7921 to fix it.
msg320520 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-26 22:19
See also bpo-33966: test_multiprocessing_spawn.WithProcessesTestPool.test_traceback() leaks 4 handles on Windows.
msg320565 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-27 09:40
New changeset 2cc9d21fffb8146d30e6fb4221e32410ba4b4ab7 by Victor Stinner in branch 'master':
bpo-33929: multiprocessing: fix handle leak on race condition (GH-7921)
https://github.com/python/cpython/commit/2cc9d21fffb8146d30e6fb4221e32410ba4b4ab7
msg320566 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-27 09:43
Python 2.7 is not affected: it doesn't use DuplicateHandle() with DUPLICATE_CLOSE_SOURCE.
msg320569 - (view) Author: miss-islington (miss-islington) Date: 2018-06-27 09:59
New changeset 8b1ebcd7cb3319273ea635df78ebf9ad40171514 by Miss Islington (bot) in branch '3.7':
bpo-33929: multiprocessing: fix handle leak on race condition (GH-7921)
https://github.com/python/cpython/commit/8b1ebcd7cb3319273ea635df78ebf9ad40171514
msg320575 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-27 13:17
New changeset 7b5856e7f059748f41f37d00abc76be0807d989d by Victor Stinner in branch '3.7':
Revert "bpo-33929: multiprocessing: fix handle leak on race condition (GH-7921)" (GH-7963)
https://github.com/python/cpython/commit/7b5856e7f059748f41f37d00abc76be0807d989d
msg320576 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-27 13:18
New changeset 0aab8660cdaa540730994afbce49a146dd779bce by Victor Stinner in branch 'master':
bpo-33929: Fix regression in spawn_main() (#7962)
https://github.com/python/cpython/commit/0aab8660cdaa540730994afbce49a146dd779bce
msg320577 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-27 13:19
I was decided to only fix the bug in the master branch, since it's a minor bug (leak a few handles in a rare case), whereas the backport can introduce regressions.
https://github.com/python/cpython/pull/7921#issuecomment-400612321

The bug has been fixed in the master branch, I close the issue.

Sorry for the commit/revert in the 3.7 branch, it's an issue with miss-inlington bot (and also my fault!).
History
Date User Action Args
2018-06-27 13:19:27vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg320577

stage: patch review -> resolved
2018-06-27 13:18:41vstinnersetmessages: + msg320576
2018-06-27 13:17:38vstinnersetmessages: + msg320575
2018-06-27 10:08:52vstinnersetpull_requests: + pull_request7570
2018-06-27 09:59:47miss-islingtonsetnosy: + miss-islington
messages: + msg320569
2018-06-27 09:59:38vstinnersetpull_requests: + pull_request7569
2018-06-27 09:46:46vstinnersetpull_requests: + pull_request7568
2018-06-27 09:43:46vstinnersetmessages: + msg320566
versions: + Python 3.6, Python 3.7
2018-06-27 09:41:41miss-islingtonsetpull_requests: + pull_request7567
2018-06-27 09:40:27vstinnersetmessages: + msg320565
2018-06-26 22:19:17vstinnersetmessages: + msg320520
2018-06-26 00:11:42vstinnersetmessages: + msg320457
2018-06-26 00:10:25vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request7527
2018-06-21 11:05:06vstinnersettitle: test_multiprocessing_spawn leaks 5 handles -> test_multiprocessing_spawn: WithProcessesTestProcess.test_many_processes() leaks 5 handles on Windows
2018-06-21 11:04:48vstinnersetnosy: + pitrou, davin
2018-06-21 11:02:52vstinnercreate