Message400700
PR 28082 is a draft that implements this idea. Skipped and failed (but not successfully passed) subtests are now reported separately, as a character (sFE) or a line ("skipped", "FAIL", "ERROR"). The description of the subtest is included in the line. For example:
$ tests=.sFE ./python test_issue25894.py
sFE
======================================================================
ERROR: test_subTest (__main__.TestClass) [3] (t='E')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/serhiy/py/cpython/test_issue25894.py", line 15, in test_subTest
raise Exception('error')
^^^^^^^^^^^^^^^^^^^^^^^^
Exception: error
======================================================================
FAIL: test_subTest (__main__.TestClass) [2] (t='F')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/serhiy/py/cpython/test_issue25894.py", line 13, in test_subTest
self.fail('failed')
^^^^^^^^^^^^^^^^^^^
AssertionError: failed
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1, errors=1, skipped=1)
$ tests=.sFE ./python test_issue25894.py -v
test_subTest (__main__.TestClass) ...
test_subTest (__main__.TestClass) [1] (t='s') ... skipped 'skipped'
test_subTest (__main__.TestClass) [2] (t='F') ... FAIL
test_subTest (__main__.TestClass) [3] (t='E') ... ERROR
======================================================================
ERROR: test_subTest (__main__.TestClass) [3] (t='E')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/serhiy/py/cpython/test_issue25894.py", line 15, in test_subTest
raise Exception('error')
^^^^^^^^^^^^^^^^^^^^^^^^
Exception: error
======================================================================
FAIL: test_subTest (__main__.TestClass) [2] (t='F')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/serhiy/py/cpython/test_issue25894.py", line 13, in test_subTest
self.fail('failed')
^^^^^^^^^^^^^^^^^^^
AssertionError: failed
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1, errors=1, skipped=1)
As a side effect, the test description is also repeated for every error in the test cleanup code (in tearDown() and doCleanup()).
Similar changes should be added also in RegressionTestResult. If apply issue45057 first they will be much simpler.
Issue29152 can be related. If call addError() and addFailure() from addSubTest(), PR 28082 should be rewritten. |
|
Date |
User |
Action |
Args |
2021-08-31 08:16:13 | serhiy.storchaka | set | recipients:
+ serhiy.storchaka, pitrou, rbcollins, ezio.melotti, r.david.murray, michael.foord, martin.panter, zach.ware |
2021-08-31 08:16:13 | serhiy.storchaka | set | messageid: <1630397773.4.0.568148134648.issue25894@roundup.psfhosted.org> |
2021-08-31 08:16:13 | serhiy.storchaka | link | issue25894 messages |
2021-08-31 08:16:13 | serhiy.storchaka | create | |
|