Python test runner regrtest has many options to select tests: -u (resources), -m and --matchfile, -x (exclude), etc. Sometimes, I do a mistake and I run 0 test, but I still get a success.

Maybe regrtest should be modified to fail with an error when no test is run? At least, it would help to understand my mistake if regrtest could emit a warning in such case.

Example:
---

$ ./python  -W error  -m test test_asyncio -m test_set_default_executor_deprecation_warnings 
Run tests sequentially
0:00:00 load avg: 0.27 [1/1] test_asyncio

== Tests result: SUCCESS ==

1 test OK.

Total duration: 266 ms
Tests result: SUCCESS
---

Success ok, but is there a typo in the test method?

Another example:
---

$ ./python  -W error  -m test test_asyncio -m test_no_such_test -v
== CPython 3.8.0a0 (heads/pr/8533:50c0499950, Jul 30 2018, 12:41:53) [GCC 8.1.1 20180712 (Red Hat 8.1.1-5)]
== Linux-4.17.6-200.fc28.x86_64-x86_64-with-glibc2.26 little-endian
== cwd: /home/vstinner/prog/python/master/build/test_python_5505
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
Run tests sequentially
0:00:00 load avg: 0.31 [1/1] test_asyncio

Ran 0 tests in 0.001s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 318 ms
Tests result: SUCCESS
---

Success but no test have been executed. It's not obvious at all without the verbose mode:
---

$ ./python  -W error  -m test test_asyncio -m test_no_such_test 
Run tests sequentially
0:00:00 load avg: 0.31 [1/1] test_asyncio

== Tests result: SUCCESS ==

1 test OK.

Total duration: 260 ms
Tests result: SUCCESS
---

Do we need to output OK and SUCCESS many times? I would prefer shorter and totally different from the normal output in case of no tests ran.

I would prefer shorter and totally different from the normal output in case of no tests ran.

That's basically the whole purpose of the issue, yep ;-)

A test file that runs no test functions should pass on the buildbots. (If it imports the target file, it is not completely empty. And I can think of a use case for WIP PRs that intentionally do not run anything.)

But I also would like a visually different display, having been fooled more than once. For one thing, one can omit '(unittest.testCase)' from test class definitions ;-).

New changeset 9724348 by Victor Stinner (Pablo Galindo) in branch 'master':
bpo-34279, regrtest: Issue a warning if no tests have been executed (GH-10150)
9724348

New changeset 3600300 by Victor Stinner in branch '2.7':
bpo-34279, regrtest: Issue a warning if no tests have been executed (GH-10801)
3600300

New changeset 8a73cac by Victor Stinner in branch '3.7':
[3.7] bpo-34279: Synchronize regrtest with master (GH-10800)
8a73cac

New changeset 43d8126 by Miss Islington (bot) in branch '3.6':
[3.7] bpo-34279: Synchronize regrtest with master (GH-10800)
43d8126

Thanks Pablo! I merged your change in 2.7, 3.6, 3.7 and master branches.

To reuse the example of my first message, we now get:

vstinner@apu$ ./python -W error -m test test_asyncio -m test_no_such_test
Run tests sequentially
0:00:00 load avg: 1.21 [1/1] test_asyncio
test_asyncio run no tests

== Tests result: NO TEST RUN ==

1 test run no tests:
test_asyncio

Total duration: 347 ms
Tests result: NO TEST RUN

vstinner@apu$ echo $?
0

So the exit status is still a success (code 0), but "Tests result: NO TEST RUN" is an hint that no test has been executed.

If we run two files, but one file runs no test, the output now contains "1 test run no tests: test_sys":

$ ./python -m test test_os test_sys -m test_access -v
== CPython 3.8.0a0 (heads/platform_popen:4fcc412d3d, Nov 29 2018, 21:21:53) [GCC 8.2.1 20181105 (Red Hat 8.2.1-5)]
== Linux-4.19.3-300.fc29.x86_64-x86_64-with-glibc2.28 little-endian
== cwd: /home/vstinner/prog/python/master/build/test_python_18420
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
Run tests sequentially
0:00:00 load avg: 1.05 [1/2] test_os
test_access (test.test_os.FileTests) ... ok

Ran 1 test in 0.001s

OK
0:00:00 load avg: 1.05 [2/2] test_sys

----------------------------------------------------------------------

Ran 0 tests in 0.000s

OK
test_sys run no tests

== Tests result: SUCCESS ==

1 test OK.

1 test run no tests:
test_sys

Total duration: 240 ms
Tests result: SUCCESS

New changeset 3a8f4fe by Victor Stinner in branch 'master':
bpo-34279: regrtest consider that skipped tests are ran (GH-11132)
3a8f4fe

New changeset 5f252e1 by Miss Islington (bot) in branch '3.7':
bpo-34279: regrtest consider that skipped tests are ran (GH-11132)
5f252e1

New changeset 34b7c43 by Victor Stinner in branch '2.7':
bpo-34279: regrtest consider that skipped tests are ran (GH-11132) (GH-11158)
34b7c43