classification
Title: test_concurrent_futures stopped after 25 hours on AMD64 Windows7 SP1 3.7
Type: Stage: resolved
Components: Tests Versions: Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS
View: 37531
Assigned To: Nosy List: jkloth, vstinner
Priority: normal Keywords:

Created on 2019-06-17 08:26 by vstinner, last changed 2019-07-29 23:48 by vstinner. This issue is now closed.

Messages (3)
msg345818 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-17 08:26
AMD64 Windows7 SP1 3.7:
https://buildbot.python.org/all/#/builders/130/builds/935

0:23:27 load avg: 7.92 [412/416] test_plistlib passed -- running: test_venv (1 min 26 sec), test_concurrent_futures (23 min 26 sec)
0:23:32 load avg: 7.29 [413/416] test_argparse passed -- running: test_venv (1 min 31 sec), test_concurrent_futures (23 min 31 sec)
0:23:34 load avg: 7.29 [414/416] test_venv passed (1 min 32 sec) -- running: test_concurrent_futures (23 min 32 sec)
0:23:37 load avg: 6.71 [415/416] test_largefile passed -- running: test_concurrent_futures (23 min 35 sec)
running: test_concurrent_futures (24 min 6 sec)
running: test_concurrent_futures (24 min 36 sec)
...
running: test_concurrent_futures (25 hour 25 min)
running: test_concurrent_futures (25 hour 26 min)
running: test_concurrent_futures (25 hour 26 min)
running: test_concurrent_futures (25 hour 27 min)
running: test_concurrent_futures (25 hour 27 min)
running: test_concurrent_futures (25 hour 28 min)
program finished with exit code 1
elapsedTime=91968.468000
msg345845 - (view) Author: Jeremy Kloth (jkloth) * Date: 2019-06-17 11:05
The test run was stopped by me terminating the stuck processes on the buildbot worker itself.  This happens approximately once a month or so.

As of late, its been stuck in test_concurrent_futures, but it can get stuck in any test that uses multiprocessing (the spawned pool never terminates).
msg348703 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-07-29 23:48
The root issue will be solved by bpo-37531: ensure that regrtest kills a worker process if it takes longer than TIMEOUT seconds.
History
Date User Action Args
2019-07-29 23:48:28vstinnersetstatus: open -> closed
superseder: Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS
messages: + msg348703

resolution: duplicate
stage: resolved
2019-06-17 11:05:17jklothsetnosy: + jkloth
messages: + msg345845
2019-06-17 08:26:00vstinnercreate