classification
Title: test target of Makefile always run tests in parallel mode
Type: Stage: resolved
Components: Tests Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: vstinner Nosy List: Arfrever, vstinner
Priority: normal Keywords:

Created on 2017-08-01 18:57 by Arfrever, last changed 2019-06-24 13:34 by vstinner. This issue is now closed.

Messages (3)
msg299629 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2017-08-01 18:57
In CPython >=3.3, "test" target of Makefile calls Tools/scripts/run_tests.py script.
This script contains:

def is_multiprocess_flag(arg):
    return arg.startswith('-j') or arg.startswith('--multiprocess')
...
def main(regrtest_args):
...
    if threading and not any(is_multiprocess_flag(arg) for arg in regrtest_args):
        args.extend(['-j', '0'])  # Use all CPU cores


In order to run tests sequentially, in CPython 3.5 and older branches, it is possible to set EXTRATESTOPTS with -j1:
    make test EXTRATESTOPTS="-j1"


In CPython >=3.6, this workaround no longer works, because this code (Lib/test/regrtest.py in 3.5):

    if ns.use_mp is not None:
        if ns.use_mp <= 0:
            # Use all cores + extras for tests that like to sleep
            ns.use_mp = 2 + (os.cpu_count() or 1)
        if ns.use_mp == 1:
            ns.use_mp = None

Was changed into (Lib/test/libregrtest/cmdline.py in 3.6 and 3.7):

    if ns.use_mp is not None:
        if ns.use_mp <= 0:
            # Use all cores + extras for tests that like to sleep
            ns.use_mp = 2 + (os.cpu_count() or 1)


Currently the only remaining ways to run tests sequentially are to locally edit Tools/scripts/run_tests.py or to not  use "test" target of Makefile and to run Lib/test/regrtest.py directly with appropriate options.
msg346400 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-24 13:32
To run tests sequentially, you can run regrtest using:

./python -m test

I don't use "make test".
msg346401 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-24 13:34
I close the issue as "not a bug". Reopen if you disagree.

IMHO it's a bad practice to run tests sequentially: a test can change th behavior of the following test. Spawning a new process per test file is safer. Even if we are working hard to fix all side effects.
History
Date User Action Args
2019-06-24 13:34:48vstinnersetstatus: open -> closed
resolution: not a bug
messages: + msg346401

stage: resolved
2019-06-24 13:32:08vstinnersetmessages: + msg346400
2017-08-01 18:57:38Arfrevercreate