On Python 2.7, listing *all* test cases with "./python -m test --list-cases" fails on the following test files:

* test_multibytecodec_support 
* test_robotparser

regrtest of Python 2.7 uses the test_main() function of each test file, if the function is defined. Whereas --list-cases is more written as a raw discovery function listing all cases which inherit from unittest.TestCase. These 3 files have base test cases which are "abstract" or configured below.

Options:

* Modify --list-cases to tolerate failures and report them at exit
* Modify these 2 tests to make them more "test discovery"-friendly
* Use test_main() function if available but modify its behaviour to only list test cases, instead of running them

I would prefer to limit changes in Python 2.7 branch, so IMHO the second option is the best: modify the 2 tests.


haypo@selma$ ./python -m test --list-cases test_multibytecodec_support
Traceback (most recent call last):
  ...
  File "/home/haypo/prog/python/2.7/Lib/test/test_multibytecodec_support.py", line 268, in __init__
    self.open_mapping_file().close() # test it to report the error early
  File "/home/haypo/prog/python/2.7/Lib/test/test_multibytecodec_support.py", line 273, in open_mapping_file
    return test_support.open_urlresource(self.mapfileurl)
AttributeError: 'TestBase_Mapping' object has no attribute 'mapfileurl'


haypo@selma$ ./python -m test --list-cases test_robotparser
Traceback (most recent call last):
  ...
  File "/home/haypo/prog/python/2.7/Lib/test/regrtest.py", line 1511, in list_cases
    suite = unittest.defaultTestLoader.loadTestsFromName(abstest)
  File "/home/haypo/prog/python/2.7/Lib/unittest/loader.py", line 103, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/home/haypo/prog/python/2.7/Lib/unittest/loader.py", line 65, in loadTestsFromModule
    tests.append(self.loadTestsFromTestCase(obj))
  File "/home/haypo/prog/python/2.7/Lib/unittest/loader.py", line 56, in loadTestsFromTestCase
    loaded_suite = self.suiteClass(map(testCaseClass, testCaseNames))
TypeError: __init__() takes exactly 6 arguments (2 given)

I have doubts that only 2 tests need to be modified. Making tests discoverable in Python 3 was a long work made by many developers in many issues, and it still is not finished. Some of changes were not trivial and needed rewriting the architecture of the tests.

Yet one option is forgot about 2.7.

Well, this issue is not about discovering all test methods, but just to not
fail with an error on: ./python -m test --list-cases

For Lib/test/test_robotparser.py: with the current design of the test file, it's not easy to fix. I tried a different approach: copy the file from the match branch and adapt it for Python 2.7.

See https://github.com/python/cpython/pull/2546

New changeset 668489a6d5a5b124aea35820ca90cb5e3b2b3ecd by Victor Stinner in branch '2.7':
bpo-30759: Copy test_robotparser from master (#2546)
https://github.com/python/cpython/commit/668489a6d5a5b124aea35820ca90cb5e3b2b3ecd

Hum, it still doesn't work. With my latest PR, "./python -m test --list-cases" now work on all tests of Python 2.7:

https://github.com/python/cpython/pull/2582

New changeset 8767de2f776e0c8c7404680cdacad83e5d902955 by Victor Stinner in branch '2.7':
bpo-30759: regrtest: list_cases() now unload modules (#2582)
https://github.com/python/cpython/commit/8767de2f776e0c8c7404680cdacad83e5d902955

Ok, this one is now fixed ;-)