Author Kit Yan Choi
Recipients Kit Choi, Kit Yan Choi, terry.reedy
Date 2019-09-27.21:15:38
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1569618938.92.0.581923185772.issue38296@roundup.psfhosted.org>
In-reply-to
Content
For your test:

class T(unittest.TestCase):
    def test_f(self): raise TypeError()

If you run this test with unittest test runner, you should get this result:

E
======================================================================
ERROR: test_f (test_main.T)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_main.py", line 5, in test_f
    def test_f(self): raise TypeError()
TypeError

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)


I expect to get this behaviour even if the test is decorated with unittest.expectedFailure. However, currently we get a success.


Scenario:


You create a class named Duck with a method "quack".  Then you added a test, and test needs to call Duck.quack.

Later on for whatever reason, you need to decorate the test with expectedFailure.  The test passes with the expected failure.

Then you rename the "quack" method to "walk", but you forget to update the test.  Now the test is actually failing with an AttributeError, but you won't notice it because expectedFailure silences it.

In this scenario, it is important to differentiate a "test error" and a "test failure". A test has four status: success, failure, error, skipped.  I expect unittest.expectedFailure to make "failure" a "success" and a "success" a "failure", and it should leave "error" and "skipped" unchanged.

Please consider reopening this issue.
History
Date User Action Args
2019-09-27 21:15:38Kit Yan Choisetrecipients: + Kit Yan Choi, terry.reedy, Kit Choi
2019-09-27 21:15:38Kit Yan Choisetmessageid: <1569618938.92.0.581923185772.issue38296@roundup.psfhosted.org>
2019-09-27 21:15:38Kit Yan Choilinkissue38296 messages
2019-09-27 21:15:38Kit Yan Choicreate