Message251419
The Lib/test/regrtest.py file became a monster. It's very long, it uses a lot of global variables. It's hard to review and hard to maintain. I propose to:
- split single file regrtest.py into multiple files in a new Lib/test/regrtest/ package
- refactor the test runner as a class to avoid global variables
- refactor the multiprocessing test runner as a class in a separated file
Attached patch implements this idea.
The following commands still work:
./python -m test [options] [args]
./python -m test.regrtest [options] [args]
But the following command doesn't work anymore:
./python Lib/test/regrtest.py [options] [args]
If regrtest.py is executed directly on buildbots, these buildbots should be modified first.
The "./python -m test.regrtest" command should work on all Python versions (Python 2 and Python 3) and so should be preferred. The "./python -m test" command only works on Python 3 (thanks to the new __main__.py feature).
The change adds a new feature: it now displays name of concurrent tests running since 30 seconds or longer when the multiprocessing test runner is used (-j command line option). Example:
...
[240/399] test_uu
[241/399] test_urllib_response
[242/399] test_struct
[243/399] test_descrtut
[244/399] test_threadedtempfile
[245/399] test_tracemalloc -- running: test_concurrent_futures (30 sec)
[246/399] test_dbm_dumb -- running: test_concurrent_futures (30 sec)
[247/399] test_codeop -- running: test_concurrent_futures (30 sec)
...
[395/399/1] test_asyncio -- running: test_multiprocessing_fork (40 sec), test_multiprocessing_spawn (44 sec)
[396/399/1] test_faulthandler -- running: test_multiprocessing_fork (50 sec), test_multiprocessing_spawn (54 sec)
[397/399/1] test_multiprocessing_fork (52 sec) -- running: test_multiprocessing_spawn (56 sec)
[398/399/1] test_multiprocessing_spawn (68 sec) -- running: test_multiprocessing_forkserver (39 sec)
[399/399/1] test_multiprocessing_forkserver (50 sec)
I want this feature to analysis why more and more buildbots fail with a timeout without saying which test was running (well, I suspect multiprocessing tests...).
Note: faulthandler can show where regrtest is blocked, but not when the multiprocessing test runner is used. And sometimes the process is killed by the buildbot, not by faulthandler :-/
Another minor new feature: on CTRL+c, it shows which tests are running when the multiprocessing test runner is used. Example:
[ 38/399] test_dummy_thread
[ 39/399] test_codecmaps_jp
[ 40/399] test_future5
^C
Waiting for test_scope, test_decimal, test_memoryview, test_heapq, test_unicodedata, test_trace, test_threadsignals, test_cgitb, test_runpy, test_cmd_line_script
Other changes:
* Show test timing when a test runs longer than 30 seconds
* Don't make __file__ absolute, findtestdir() calls os.path.abspath() instead. Remove these lines:
__file__ = os.path.abspath(__file__)
assert __file__ == os.path.abspath(sys.argv[0])
* print() is now called wih flush=True (it didn't check all calls, only the major calls), remove sys.stdout.flush() and sys.stdout.flush()
* A lot of refactoring. Sorry, I didn't take notes for each change.
I fear that test_regrtest has a small code coverage. I only tested major options, I didn't test -R for example.
Note: I don't understand how the --single option works when regrtest is not run from the Python source code directory. A temporary directory, so the pynexttest file is removed after its creation, no? If it doesn't make sense to use --single outside Python directory, maybe an error should be raised? |
|
Date |
User |
Action |
Args |
2015-09-23 10:12:35 | vstinner | set | recipients:
+ vstinner, ezio.melotti, serhiy.storchaka |
2015-09-23 10:12:34 | vstinner | set | messageid: <1443003154.43.0.463395539822.issue25220@psf.upfronthosting.co.za> |
2015-09-23 10:12:34 | vstinner | link | issue25220 messages |
2015-09-23 10:12:27 | vstinner | create | |
|