This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_multiprocessing_spawn leaks QueueManager dangling processes
Type: Stage: patch review
Components: Tests Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: erlendaasland, pablogsal, vstinner
Priority: normal Keywords: patch

Created on 2022-03-21 03:01 by vstinner, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
manager_timeout.patch vstinner, 2022-03-21 03:01
Pull Requests
URL Status Linked Edit
PR 32112 open vstinner, 2022-03-25 11:35
Messages (4)
msg415650 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-21 03:01
Apply attached manager_timeout.patch and run the command:

    ./python -m test --fail-env-changed test_multiprocessing_spawn -v -m test_rapid_restart -F -j4

If you fail to reproduce the issue, replace -j4 with -j10 or use a larger number of concurrent jobs.

Output:
---
(...)
0:00:00 load avg: 2.85 Run tests in parallel using 4 child processes
0:00:06 load avg: 3.02 [  1/1] test_multiprocessing_spawn failed (env changed)
test_rapid_restart (test.test_multiprocessing_spawn.WithManagerTestManagerRestart) ... ok

test_rapid_restart (test.test_multiprocessing_spawn.WithProcessesTestManagerRestart) ... ok
Warning -- Dangling processes: {<SpawnProcess name='QueueManager-5' pid=438355 parent=438255 stopped exitcode=-SIGTERM>, <SpawnProcess name='QueueManager-7' pid=438395 parent=438255 stopped exitcode=-SIGTERM>}

test_rapid_restart (test.test_multiprocessing_spawn.WithThreadsTestManagerRestart) ... ok
Warning -- Dangling processes: {<SpawnProcess name='QueueManager-8' pid=438423 parent=438255 stopped exitcode=-SIGTERM>, <SpawnProcess name='QueueManager-9' pid=438456 parent=438255 stopped exitcode=-SIGTERM>}
Warning -- Dangling processes: {<SpawnProcess name='QueueManager-8' pid=438423 parent=438255 stopped exitcode=-SIGTERM>, <SpawnProcess name='QueueManager-9' pid=438456 parent=438255 stopped exitcode=-SIGTERM>}

----------------------------------------------------------------------
Ran 3 tests in 5.404s

OK
Kill <TestWorkerProcess #2 running test=test_multiprocessing_spawn pid=438256 time=6.9 sec> process group
Kill <TestWorkerProcess #3 running test=test_multiprocessing_spawn pid=438259 time=6.9 sec> process group
Kill <TestWorkerProcess #4 running test=test_multiprocessing_spawn pid=438260 time=6.9 sec> process group

== Tests result: ENV CHANGED ==

1 test altered the execution environment:
    test_multiprocessing_spawn

(...)
---


Examples of similar old issues:

* bpo-39932 at https://bugs.python.org/issue39932#msg363925 : <ForkServerProcess(QueueManager-2036, stopped[1])>
* bpo-38447: <SpawnProcess(QueueManager-1576, stopped)>
* bpo-38796: <SpawnProcess name='QueueManager-327' pid=60315 parent=59088 started>
* bpo-34150: <SpawnProcess(QueueManager-181, stopped)>
* bpo-31069 at https://bugs.python.org/issue31069#msg301339 : <ForkServerProcess(QueueManager-465, stopped)>

See also the old fixed bpo-31234 "Make support.threading_cleanup() stricter".
msg415651 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-21 03:06
BaseManager._finalize_manager() timeout should be configurable, the test should use the same timeout than support.wait_process() (SHORT_TIMEOUT), and maybe the default timeout should be increased?

I propose to use the same timeout for the two process.join(timeout=xxx) calls (before/after terminate).
msg415652 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-21 03:10
Recent failures.

AMD64 FreeBSD Non-Debug 3.x:
https://buildbot.python.org/all/#/builders/172/builds/1654
---
Warning -- Dangling processes: {<SpawnProcess name='QueueManager-335' pid=15699 parent=14154 stopped exitcode=-SIGTERM>}
Warning -- Dangling processes: {<SpawnProcess name='QueueManager-335' pid=15699 parent=14154 stopped exitcode=-SIGTERM>}

1 test altered the execution environment:
    test_multiprocessing_spawn
---
msg415996 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-25 11:40
> BaseManager._finalize_manager() timeout should be configurable

I proposed GH-32112 for that.
History
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91231
2022-03-25 11:40:41vstinnersetmessages: + msg415996
2022-03-25 11:35:14vstinnersetstage: patch review
pull_requests: + pull_request30188
2022-03-21 03:10:08vstinnersetmessages: + msg415652
2022-03-21 03:06:58vstinnersetmessages: + msg415651
2022-03-21 03:04:33vstinnersetnosy: + pablogsal, erlendaasland
2022-03-21 03:01:30vstinnercreate