classification
Title: test_multiprocessing_forkserver leaked [1, 2, 1] memory blocks on x86 Gentoo Refleaks 3.x
Type: Stage: resolved
Components: Versions:
process
Status: closed Resolution: duplicate
Dependencies: Superseder: test_multiprocessing_spawn leaked [1, 2, 1] memory blocks on AMD64 Windows8.1 Refleaks 3.7
View: 33735
Assigned To: Nosy List: vstinner
Priority: normal Keywords:

Created on 2018-06-27 22:44 by vstinner, last changed 2018-07-03 11:40 by vstinner. This issue is now closed.

Messages (4)
msg320636 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-27 22:44
http://buildbot.python.org/all/#/builders/1/builds/267

1:47:06 load avg: 4.77 [161/417/1] test_multiprocessing_forkserver failed (16 min 32 sec) -- running: test_support (56 sec 282 ms)
beginning 6 repetitions
123456
......
test_multiprocessing_forkserver leaked [1, 2, 1] memory blocks, sum=4
(...)
Re-running failed tests in verbose mode
Re-running test 'test_multiprocessing_forkserver' in verbose mode
(...)
OK (skipped=27)
msg320656 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-28 10:02
> http://buildbot.python.org/all/#/builders/1/builds/267

Oh, this is: x86 Gentoo Refleaks 3.x.

--

x86 Gentoo Refleaks 3.7 also leaked:
http://buildbot.python.org/all/#/builders/114/builds/150

test_multiprocessing_forkserver leaked [1, 2, 1] memory blocks, sum=4
Re-running failed tests in verbose mode
Re-running test 'test_multiprocessing_forkserver' in verbose mode
test_multiprocessing_forkserver leaked [21, 2, 1] memory blocks, sum=24
test_multiprocessing_forkserver leaked [2, 0, 0] file descriptors, sum=2
msg320965 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-03 11:34
Oh. I have a very good news: this issue is a duplicate of bpo-33735 and I just identified and fixed bpo-33735!
msg320967 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-03 11:40
Hum. Sadly my commit 23401fb960bb94e6ea62d2999527968d53d3fc65 is not enough, the test still fails randomly:

pydev@stormageddon ~/cpython $ ./python -m test test_multiprocessing_forkserver -m test_imap_unordered -R 3:3 -F
Run tests sequentially
0:00:00 load avg: 5.79 [  1] test_multiprocessing_forkserver
beginning 6 repetitions
123456
......
0:00:28 load avg: 6.98 [  2] test_multiprocessing_forkserver
beginning 6 repetitions
123456
......
0:00:59 load avg: 7.69 [  3] test_multiprocessing_forkserver -- test_multiprocessing_forkserver passed in 30 sec 717 ms
beginning 6 repetitions
123456
......
0:01:28 load avg: 8.63 [  4] test_multiprocessing_forkserver
beginning 6 repetitions
123456
......
test_multiprocessing_forkserver leaked [2, 1, 1] memory blocks, sum=4
test_multiprocessing_forkserver failed

== Tests result: FAILURE ==

3 tests OK.

1 test failed:
    test_multiprocessing_forkserver

Total duration: 1 min 55 sec
Tests result: FAILURE


The test failed when the system load was high (8.63).

I hope that in pratice, my change will be enough to get less random failures. The "Re-run in verbose mode" on the "idle" buildbot should make the bug disappear.
History
Date User Action Args
2018-07-03 11:40:06vstinnersetmessages: + msg320967
2018-07-03 11:34:44vstinnersetstatus: open -> closed
resolution: duplicate
messages: + msg320965

superseder: test_multiprocessing_spawn leaked [1, 2, 1] memory blocks on AMD64 Windows8.1 Refleaks 3.7
stage: resolved
2018-06-28 10:02:06vstinnersetmessages: + msg320656
2018-06-27 22:44:57vstinnercreate