New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
unittest TestResult wasSuccessful returns True when there are unexpected successes #64364
Comments
Python 3.3.3+ (3.3:28337a8fb502+, Jan 7 2014, 01:32:44)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import unittest
>>> r = unittest.result.TestResult()
>>> r.wasSuccessful()
True
>>> r.addUnexpectedSuccess("weird!")
>>> r.wasSuccessful()
True An unexpected success is not a good thing and indicates a problem. the wasSuccessful() method should include a check for len(self.unexpectedSuccesses) == 0 as part of its condition. |
somewhat related - https://code.google.com/p/unittest-ext/issues/detail?id=22 |
I'm pretty sure this has been debated before (and the status quo is the result). Trying to find the issue. |
Hmmm... TestTools has wasSuccessful return False on an unexpected success [1] and I can't find an issue for any previous discussion. I don't use unexpected success, so I have no particular horse in this race but it seems more logical that wasSuccessful should return False when there are unexpected successes. [1] https://code.launchpad.net/~jml/testtools/unexpected-success-2/+merge/42050 |
I'm not comfortable changing this for 2.7 or 3.3 in case some code is unfortunately is depending on this behavior. But as it is it does seem like the kind of thing that can hide problems (tests that are passing that are not expected to). Here's a patch for 3.4 (minus the documentation update that'll need to explicitly mention this with a versionchanged::). |
Since I too was bitten by this issue, I'd like to support Gregory's With the current behaviour, we could see (and I *have* seen) scenarios
In addition to this rationale, another (weaker) reason to change the
|
New changeset 1e75ab9fd760 by Gregory P. Smith in branch 'default': |
See also bpo-22815. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: