classification
Title: test_multiprocessing_fork: dangling threads warning
Type: Stage: resolved
Components: Tests Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, vstinner
Priority: normal Keywords: patch

Created on 2018-05-28 23:03 by vstinner, last changed 2018-11-28 00:44 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
mp_debug.patch vstinner, 2018-11-27 23:38
Pull Requests
URL Status Linked Edit
PR 10755 merged vstinner, 2018-11-27 23:28
PR 10756 merged miss-islington, 2018-11-28 00:14
PR 10757 merged miss-islington, 2018-11-28 00:14
Messages (6)
msg317948 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-05-28 23:03
AMD64 FreeBSD 10.x Shared 3.7:

http://buildbot.python.org/all/#/builders/124/builds/327

== CPU count: 2
Run tests in parallel using 4 child processes
(...)
0:02:09 load avg: 3.70 [138/415/1] test_multiprocessing_fork failed (env changed) (108 sec) -- running: test_concurrent_futures (124 sec)
(...)
test_starmap (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_starmap_async (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_terminate (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_traceback (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok
test_wrapped_exception (test.test_multiprocessing_fork.WithThreadsTestPool) ... ok

Warning -- Dangling threads: {<DummyProcess(Thread-158, started daemon 34498362368)>}
msg330560 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-27 23:38
The bug is still alive:
https://buildbot.python.org/all/#/builders/168/builds/294

I found a bug using attached mp_debug.patch.

I wrote PR 10755 to try to fix it.

I have access to FreeBSD CURRENT buildbot. I only reproduced the bug once using:

./python -m test -v test_multiprocessing_fork -m WithThreadsTestPool --fail-env-changed -F 

I ran this command 4 times in parallel in 4 terminals, but I failed to reliably reproduce the bug. Even if I stress the computer using "./python -m test -j4 -r" run in different shell, or using my "system_load.py 3" script, I fail to trigger the bug.
msg330565 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-28 00:14
New changeset b7278736b3ae158a7738057e3045bc767ced019e by Victor Stinner in branch 'master':
bpo-33676: Fix dangling thread in _test_multiprocessing (GH-10755)
https://github.com/python/cpython/commit/b7278736b3ae158a7738057e3045bc767ced019e
msg330566 - (view) Author: miss-islington (miss-islington) Date: 2018-11-28 00:30
New changeset 80db40cdd6a58c9d66694a43ee9a7c06ab958373 by Miss Islington (bot) in branch '3.7':
bpo-33676: Fix dangling thread in _test_multiprocessing (GH-10755)
https://github.com/python/cpython/commit/80db40cdd6a58c9d66694a43ee9a7c06ab958373
msg330567 - (view) Author: miss-islington (miss-islington) Date: 2018-11-28 00:37
New changeset dd5293871703e6a12ffdde14eeaa86a73b7b0d99 by Miss Islington (bot) in branch '3.6':
bpo-33676: Fix dangling thread in _test_multiprocessing (GH-10755)
https://github.com/python/cpython/commit/dd5293871703e6a12ffdde14eeaa86a73b7b0d99
msg330568 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-28 00:44
I *hope* that it will be enough :-/
History
Date User Action Args
2018-11-28 00:44:38vstinnersetstatus: open -> closed
versions: + Python 3.6, Python 3.8
messages: + msg330568

resolution: fixed
stage: patch review -> resolved
2018-11-28 00:37:28miss-islingtonsetmessages: + msg330567
2018-11-28 00:30:37miss-islingtonsetnosy: + miss-islington
messages: + msg330566
2018-11-28 00:14:53miss-islingtonsetpull_requests: + pull_request10005
2018-11-28 00:14:44miss-islingtonsetstage: resolved -> patch review
pull_requests: + pull_request10004
2018-11-28 00:14:36vstinnersetmessages: + msg330565
2018-11-27 23:38:37vstinnersetstatus: closed -> open
files: + mp_debug.patch
messages: + msg330560

keywords: + patch
resolution: out of date -> (no value)
2018-11-27 23:28:29vstinnersetpull_requests: + pull_request10003
2018-09-19 23:57:36vstinnersetstatus: open -> closed
resolution: out of date
stage: resolved
2018-05-28 23:03:48vstinnercreate