This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients ezio.melotti, serhiy.storchaka, vstinner
Date 2015-09-23.10:12:14
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
The Lib/test/ file became a monster. It's very long, it uses a lot of global variables. It's hard to review and hard to maintain. I propose to:

- split single file into multiple files in a new Lib/test/regrtest/ package
- refactor the test runner as a class to avoid global variables
- refactor the multiprocessing test runner as a class in a separated file

Attached patch implements this idea.

The following commands still work:

   ./python -m test [options] [args]
   ./python -m test.regrtest [options] [args]

But the following command doesn't work anymore:

   ./python Lib/test/ [options] [args]

If is executed directly on buildbots, these buildbots should be modified first.

The "./python -m test.regrtest" command should work on all Python versions (Python 2 and Python 3) and so should be preferred. The "./python -m test" command only works on Python 3 (thanks to the new feature).

The change adds a new feature: it now displays name of concurrent tests running since 30 seconds or longer when the multiprocessing test runner is used (-j command line option). Example:

    [240/399] test_uu
    [241/399] test_urllib_response
    [242/399] test_struct
    [243/399] test_descrtut
    [244/399] test_threadedtempfile
    [245/399] test_tracemalloc -- running: test_concurrent_futures (30 sec)
    [246/399] test_dbm_dumb -- running: test_concurrent_futures (30 sec)
    [247/399] test_codeop -- running: test_concurrent_futures (30 sec)
    [395/399/1] test_asyncio -- running: test_multiprocessing_fork (40 sec), test_multiprocessing_spawn (44 sec)
    [396/399/1] test_faulthandler -- running: test_multiprocessing_fork (50 sec), test_multiprocessing_spawn (54 sec)
    [397/399/1] test_multiprocessing_fork (52 sec) -- running: test_multiprocessing_spawn (56 sec)
    [398/399/1] test_multiprocessing_spawn (68 sec) -- running: test_multiprocessing_forkserver (39 sec)
    [399/399/1] test_multiprocessing_forkserver (50 sec)

I want this feature to analysis why more and more buildbots fail with a timeout without saying which test was running (well, I suspect multiprocessing tests...).

Note: faulthandler can show where regrtest is blocked, but not when the multiprocessing test runner is used. And sometimes the process is killed by the buildbot, not by faulthandler :-/

Another minor new feature: on CTRL+c, it shows which tests are running when the multiprocessing test runner is used. Example:

    [ 38/399] test_dummy_thread
    [ 39/399] test_codecmaps_jp
    [ 40/399] test_future5
    Waiting for test_scope, test_decimal, test_memoryview, test_heapq, test_unicodedata, test_trace, test_threadsignals, test_cgitb, test_runpy, test_cmd_line_script

Other changes:

* Show test timing when a test runs longer than 30 seconds

* Don't make __file__ absolute, findtestdir() calls os.path.abspath() instead. Remove these lines:

    __file__ = os.path.abspath(__file__)
    assert __file__ == os.path.abspath(sys.argv[0])

* print() is now called wih flush=True (it didn't check all calls, only the major calls), remove sys.stdout.flush() and  sys.stdout.flush()

* A lot of refactoring. Sorry, I didn't take notes for each change.

I fear that test_regrtest has a small code coverage. I only tested major options, I didn't test -R for example.

Note: I don't understand how the --single option works when regrtest is not run from the Python source code directory. A temporary directory, so the pynexttest file is removed after its creation, no? If it doesn't make sense to use --single outside Python directory, maybe an error should be raised?
Date User Action Args
2015-09-23 10:12:35vstinnersetrecipients: + vstinner, ezio.melotti, serhiy.storchaka
2015-09-23 10:12:34vstinnersetmessageid: <>
2015-09-23 10:12:34vstinnerlinkissue25220 messages
2015-09-23 10:12:27vstinnercreate