classification
Title: In unittest.TestResult .startTestRun() and .stopTestRun() methods don't work
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ezio.melotti, michael.foord, osdiab, pitrou, py.user
Priority: normal Keywords:

Created on 2013-08-27 06:48 by py.user, last changed 2015-11-18 01:34 by osdiab.

Messages (3)
msg196266 - (view) Author: py.user (py.user) * Date: 2013-08-27 06:48
http://docs.python.org/3/library/unittest.html#unittest.TestResult.startTestRun
http://docs.python.org/3/library/unittest.html#unittest.TestResult.stopTestRun



result.py:

#!/usr/bin/env python3

import unittest

class Test(unittest.TestCase):
    def test_1(self):
        print('test_1')

    def test_2(self):
        print('test_2')
        self.fail('msg')
    
class Result(unittest.TestResult):
    def startTestRun(self, test):
        print('starttestrun', test)

    def stopTestRun(self, test):
        print('stoptestrun', test)

    def startTest(self, test):
        print('starttest', test)

    def stopTest(self, test):
        print('stoptest', test)

result = Result()
suite = unittest.defaultTestLoader.loadTestsFromTestCase(Test)
suite.run(result)

print(result)



output:

[guest@localhost result]$ ./result.py 
starttest test_1 (__main__.Test)
test_1
stoptest test_1 (__main__.Test)
starttest test_2 (__main__.Test)
test_2
stoptest test_2 (__main__.Test)
<__main__.Result run=0 errors=0 failures=1>
[guest@localhost result]$



I tried also print messages to a file - same thing
msg196287 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2013-08-27 13:30
It is the TextTestRunner that calls startTestRun and stopTestRun not the suite. 

The documentation could be improved to make that clear *or* the call could be moved into the TestSuite as not everybody uses the TextTestRunner. That would be a change in behaviour though.
msg254826 - (view) Author: Omar Diab (osdiab) Date: 2015-11-18 01:34
I found out about this while writing my own test runner, and investigated it in an answer to a StackOverflow question here: https://stackoverflow.com/questions/32920025/how-do-i-use-unittest-testresult/33770125#33770125

The startTestRun() method is supposed to be called at the beginning of a test run. That mostly works, but if you run a TestCase.run() method—or TestSuite.run(), because it just calls TestCase.run()—it will only call that method if you don't provide a TestResult instance. That means the lifecycle method is useless in those cases, because the default TestResult.startTestRun() method is a no-op.

This is how unittest.TestCase.run() starts:

def run(self, result=None):
    orig_result = result
    if result is None:
        result = self.defaultTestResult()
        startTestRun = getattr(result, 'startTestRun', None)
        if startTestRun is not None:
            startTestRun()
    # ...

The branch for calling startTestRun() is inside the `if result is None` check, making it never get called with an implementation of startTestRun().

This is currently mitigatable by just calling startTestRun() manually. But the documentation makes it seem like this lifecycle method should be called automatically, when it is not.
History
Date User Action Args
2015-11-18 01:34:52osdiabsetnosy: + osdiab
messages: + msg254826
2013-08-27 13:30:35michael.foordsetmessages: + msg196287
2013-08-27 13:30:11serhiy.storchakasetnosy: + pitrou, ezio.melotti
2013-08-27 12:52:03r.david.murraysetnosy: + michael.foord
2013-08-27 06:48:50py.usercreate