classification
Title: test_multiprocessing_fork leaked [0, 2, 0] file descriptors on aarch64 RHEL8 Refleaks 3.7 buildbot
Type: Stage: resolved
Components: Tests Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: vstinner
Priority: normal Keywords: patch

Created on 2020-03-11 11:52 by vstinner, last changed 2020-04-23 22:20 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19690 merged vstinner, 2020-04-23 21:40
Messages (6)
msg363909 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-11 11:52
aarch64 RHEL8 Refleaks 3.7:
https://buildbot.python.org/all/#/builders/620/builds/20

test_multiprocessing_fork leaked [0, 2, 0] file descriptors, sum=2

0:40:22 load avg: 0.93 Re-running failed tests in verbose mode
0:40:22 load avg: 0.93 Re-running test_multiprocessing_fork in verbose mode
test_multiprocessing_fork leaked [2, 0, 0] file descriptors, sum=2
msg363925 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-11 14:42
Another example on PPC64LE RHEL8 Refleaks 3.7:
https://buildbot.python.org/all/#/builders/431/builds/29

Warning -- Dangling processes: {<ForkServerProcess(QueueManager-2036, stopped[1])>}
Warning -- Dangling processes: {<ForkServerProcess(QueueManager-2036, stopped[1])>}

test_multiprocessing_spawn leaked [0, 2, 0] file descriptors, sum=2

Re-running test_multiprocessing_forkserver in verbose mode

test_multiprocessing_forkserver leaked [2, 0, 0] file descriptors, sum=2
msg364366 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-16 22:21
Bug seen on Pytho 3.7 branch, on Fedora Stable, RHEL7 and RHEL8, on different architectures.

aarch64 RHEL7 Refleaks 3.7:
https://buildbot.python.org/all/#builders/598/builds/22

PPC64LE RHEL7 Refleaks 3.7:
https://buildbot.python.org/all/#builders/414/builds/32

PPC64LE Fedora Stable Refleaks 3.7:
https://buildbot.python.org/all/#/builders/388/builds/31
msg367132 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-23 17:25
On aarch64 RHEL8 Refleaks 3.7, I managed to reproduce the issue with a single test:

./python -m test test_multiprocessing_fork -R 3:3 -m test.test_multiprocessing_fork.WithProcessesTestHeap.test_heap -v
msg367133 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-23 17:28
The following command fails randomly:

$ ./python -m test test_multiprocessing_fork -R 3:3 -m test.test_multiprocessing_fork.WithProcessesTestHeap.test_heap -v

Run 1:

test_multiprocessing_fork leaked [73, 52, 33] references, sum=158
test_multiprocessing_fork leaked [31, 23, 14] memory blocks, sum=68
test_multiprocessing_fork leaked [2, 0, 0] file descriptors, sum=2
test_multiprocessing_fork failed

Run 2:

1 test OK.

Run 6:

test_multiprocessing_fork leaked [0, 2, 0] file descriptors, sum=2
test_multiprocessing_fork failed
msg367154 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-23 22:20
New changeset 857d573257ab150d6bea666354312a5fd284b171 by Victor Stinner in branch '3.7':
bpo-39932: Fix multiprocessing test_heap() (GH-19690)
https://github.com/python/cpython/commit/857d573257ab150d6bea666354312a5fd284b171
History
Date User Action Args
2020-04-23 22:20:36vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-04-23 22:20:04vstinnersetmessages: + msg367154
2020-04-23 21:40:00vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request19010
2020-04-23 17:28:45vstinnersetmessages: + msg367133
2020-04-23 17:25:13vstinnersetmessages: + msg367132
2020-03-16 22:21:54vstinnersetmessages: + msg364366
2020-03-11 14:42:09vstinnersetmessages: + msg363925
2020-03-11 11:52:10vstinnercreate