This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: unittest load_tests protocol not working as documented
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ezio.melotti, michael.foord, rbcollins, stefan
Priority: normal Keywords:

Created on 2015-10-30 21:15 by stefan, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
unittest_bug.tgz stefan, 2015-10-30 21:15
Messages (5)
msg253757 - (view) Author: Stefan Seefeld (stefan) * (Python committer) Date: 2015-10-30 21:15
As described in the README contained in the attached tarball, I'm observing wrong behavior. I have based this code on my understanding of https://docs.python.org/2/library/unittest.html#load-tests-protocol, but the effect isn't as expected (I see duplicate appearances of tests whenever I use the load_tests() mechanism.)
msg258716 - (view) Author: Stefan Seefeld (stefan) * (Python committer) Date: 2016-01-20 21:11
Hi, I'm investigating more issues related to test loading, and thus I have discovered issue #16662.

I have found quite a number of inconsistencies and bugs in the loading of tests. But without getting a sense of what the supposed behavior is I find it difficult to come up with workarounds and fixes.

Is there a place where these issues can be discussed (rather than just looking at each bug individually) ?

I'd ultimately like to be able to invoke

  `python -m unittest my.test.subpackage` 

and have unittest pick up all the tests within that recursively, with and without load_tests() functions.

(On a tangential note, I would also like to have a mode where the found tests are listed without being executed. I have hacked a pseudo-TestRunner that does that, but I'm not sure this is the best approach.)

Is there any place where the bigger picture can be discussed ?

Thanks,
msg258747 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2016-01-21 09:18
To find and run tests recursively from a test package you want test discovery rather than (necessarily) load_tests.

For a bug report, please describe the behaviour you expect and the behaviour you're seeing instead.
msg258766 - (view) Author: Stefan Seefeld (stefan) * (Python committer) Date: 2016-01-21 20:15
I believe what I actually want is for the discovery mechanism to be fully implicit. It turns out that already *almost* works right now.

What doesn't work (and what this bug report really was about initially), is the use of the 'discover' command with the '-p "*.py"' argument, which for some reason makes certain tests (all ?) count twice. It looks like packages are visited twice, once as modules, and once via their contained '__init__.py' file...

(For the implicit discovery to work better, I believe, the discovery-specific options need to be made available through the main parser, so they can be used even without the 'discover' command.)
msg261716 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2016-03-14 02:37
What version of python are you testing with? unittest 2.7 has a number of bugs vis-a-vis namespaces and discovery. If you're testing with less than 3.5, or perhaps 3.6, please try with unittest2, which has the same fixes as the stdlib unittest.
History
Date User Action Args
2022-04-11 14:58:23adminsetgithub: 69706
2016-03-14 02:37:06rbcollinssetmessages: + msg261716
2016-01-21 20:15:55stefansetmessages: + msg258766
2016-01-21 09:18:19michael.foordsetmessages: + msg258747
2016-01-20 21:11:21stefansetmessages: + msg258716
2016-01-01 02:36:03ezio.melottisetcomponents: + Library (Lib)
versions: + Python 3.5, Python 3.6, - Python 3.4
2015-11-06 21:43:40terry.reedysetnosy: + rbcollins, ezio.melotti, michael.foord
2015-10-30 21:15:02stefancreate