I created a list of all test cases using:

./python -m test --list-cases > all_cases

The list contains 29,569 test cases. Sadly, the set().__contains__ matcher of my PR 4421 cannot be taken because test_json produces two test cases called "json". IMHO it's a bug in test_json, all test identifiers should be "fully qualified". test_builtin has a similar issue: it produces identifiers like "builtins.bin" instead of "test.test_builtin.builtins.bin".

If I removed the two "json" from all_cases, "./python -u -m test.bisect -i all_cases -v" takes less than one second before running the first test method, and so it's fast enough.

With PR 4420, it takes around 17 seconds before running the first test. The compilation of the giant regex and matching test identifiers are slow. My PR has the same bad performance when the regex path is taken.