Author vstinner
Recipients vstinner
Date 2015-09-30.20:36:02
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1443645362.81.0.234212686686.issue25285@psf.upfronthosting.co.za>
In-reply-to
Content
Currently, regrtest ignores the -j1 parameter. For -jN, the multiprocess mode is only enabled for N != 1. I propose to modify regrtest to run tests in subprocesses when -j1 is used. It would solve the issue #18906.

I also want to run tests in subprocesses in buildbot to better isolate tests. Running all tests in the same process, sequentially, cause various issues:

* various kinds of side effects
* memory leaks
* random failures (unreliable tests)

"make buildbot" already runs regrtest with -j1. I propose to also modify Tools/buildbot/test.bat to use run tests with -j1 by default on Windows.

I fixed regrtest to setup tests using the same code for classic mode and multiprocess mode: slaves (child processes running tests for regrtest -jN) now inherit
--memlimit/-M, --threshold/-t and --nowindows/-n options.

Only 3 remaining functions are incompatible with -jN: --coverage/-T and --findleaks/-l.

Finally, the multiprocess mode of regrtest now displays the duration of tests which took longer than 30 seconds, and every minute, it displays the tests running since longer than 30 seconds. Example with multiple workers (-j4):
---
...
[395/399/1] test_mimetypes -- running: test_lzma (111 sec), test_multiprocessing_forkserver (224 sec)
[396/399/1] test_poplib -- running: test_lzma (111 sec), test_multiprocessing_forkserver (225 sec)
[397/399/1] test_lzma (111 sec) -- running: test_multiprocessing_forkserver (225 sec)
[398/399/1] test_range -- running: test_multiprocessing_forkserver (227 sec)
running: test_multiprocessing_forkserver (287 sec)
[399/399/1] test_multiprocessing_forkserver (340 sec)
378 tests OK.
---

Note: "python -m test -j1 -j4" uses 4 workers, only the last -jN parameter is used. Some buildbots add -jN to use more workers.
History
Date User Action Args
2015-09-30 20:36:02vstinnersetrecipients: + vstinner
2015-09-30 20:36:02vstinnersetmessageid: <1443645362.81.0.234212686686.issue25285@psf.upfronthosting.co.za>
2015-09-30 20:36:02vstinnerlinkissue25285 messages
2015-09-30 20:36:02vstinnercreate