classification
Title: unittest API for detecting test failure in cleanup/teardown
Type: enhancement Stage: needs patch
Components: Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: azsorkin, ezio.melotti, josephgordon, martin.panter, michael.foord, r.david.murray, rbcollins
Priority: normal Keywords: easy

Created on 2015-05-20 21:01 by r.david.murray, last changed 2015-12-14 02:26 by josephgordon.

Messages (5)
msg243694 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-05-20 21:01
I have a situation where it would be really helpful to know in my cleanup routine whether or not the test failed.  The situation is this: I'm running a command in a subprocess, and sometimes it writes output to stderr that I normally want to ignore.  But if the test fails, I'd like my cleanup routine (that shuts down the test subprocess) to print out the stderr, because usually the information as to what went wrong is in stderr.

I figure out this hack (for python3.4):

    for what, result in self._outcome.errors:
        if what._testMethodName == self._testMethodName and result:
            print(self.p.stderr.read().decode())

but obviously that uses non-public APIs.
msg243703 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-05-20 23:03
Can’t you use the -b or --buffer command-line option <https://docs.python.org/dev/library/unittest.html#cmdoption-unittest-b>, or equivalent option to unittest.main() or whatever?

The standard output and standard error streams are buffered during the test run. Output during a passing test is discarded. Output is echoed normally on test fail or error and is added to the failure messages.
msg243704 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-05-20 23:06
Ah sorry I didn’t see the keyword “subprocess”, so --buffer won’t work! But maybe you can dump the subprocess’s stderr to sys.stderr?
msg243720 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-05-21 03:11
Having to specify a command line option to get a normally clean test run just doesn't feel like a good API.  If there is a way to set that on the TestCase, that would be OK.

There are other reasons to want the API I suggest, though...there is at least one StackOverflow question asking about how to do it, which is where I got the clue for implementing my hack.
msg246703 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-07-13 23:31
Setting some basic design parameters here.

Its ok for a test to know if it itself has decided on some status. See e.g. testtools.expectThat for a related design thing.

Making some official API within the test itself so code after the failure can take appropriate actions seems pretty safe to me, as long as its a one-way switch - no backsies.

I think inspecting the reporter would be overly limiting on the reporter implementation, and we shouldn't do that.

Further, I think limiting the possible status's that we track to the minimum would be good here, even though its inconsistent with the current overly rich separation unittest offers.

That is, some self.status field which might even be an enum (if enum34 supports Python 2.6 for backports [as unittest is currently backported to 2.6 and up].

unset
success
unexpected success
skipped
failed
expected failure
History
Date User Action Args
2015-12-14 02:26:57josephgordonsetnosy: + josephgordon
2015-09-08 18:39:42azsorkinsetnosy: + azsorkin
2015-07-13 23:31:45rbcollinssetmessages: + msg246703
2015-05-21 03:11:04r.david.murraysetmessages: + msg243720
2015-05-20 23:06:05martin.pantersetmessages: + msg243704
2015-05-20 23:03:59martin.pantersetnosy: + martin.panter
messages: + msg243703
2015-05-20 21:01:55r.david.murraycreate