This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author serhiy.storchaka
Recipients ezio.melotti, martin.panter, michael.foord, pitrou, r.david.murray, rbcollins, serhiy.storchaka, zach.ware
Date 2021-08-31.08:16:13
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1630397773.4.0.568148134648.issue25894@roundup.psfhosted.org>
In-reply-to
Content
PR 28082 is a draft that implements this idea. Skipped and failed (but not successfully passed) subtests are now reported separately, as a character (sFE) or a line ("skipped", "FAIL", "ERROR"). The description of the subtest is included in the line. For example:

$ tests=.sFE ./python test_issue25894.py 
sFE
======================================================================
ERROR: test_subTest (__main__.TestClass) [3] (t='E')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/test_issue25894.py", line 15, in test_subTest
    raise Exception('error')
    ^^^^^^^^^^^^^^^^^^^^^^^^
Exception: error

======================================================================
FAIL: test_subTest (__main__.TestClass) [2] (t='F')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/test_issue25894.py", line 13, in test_subTest
    self.fail('failed')
    ^^^^^^^^^^^^^^^^^^^
AssertionError: failed

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1, errors=1, skipped=1)

$ tests=.sFE ./python test_issue25894.py -v
test_subTest (__main__.TestClass) ... 
  test_subTest (__main__.TestClass) [1] (t='s') ... skipped 'skipped'
  test_subTest (__main__.TestClass) [2] (t='F') ... FAIL
  test_subTest (__main__.TestClass) [3] (t='E') ... ERROR

======================================================================
ERROR: test_subTest (__main__.TestClass) [3] (t='E')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/test_issue25894.py", line 15, in test_subTest
    raise Exception('error')
    ^^^^^^^^^^^^^^^^^^^^^^^^
Exception: error

======================================================================
FAIL: test_subTest (__main__.TestClass) [2] (t='F')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/test_issue25894.py", line 13, in test_subTest
    self.fail('failed')
    ^^^^^^^^^^^^^^^^^^^
AssertionError: failed

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1, errors=1, skipped=1)

As a side effect, the test description is also repeated for every error in the test cleanup code (in tearDown() and doCleanup()).

Similar changes should be added also in RegressionTestResult. If apply issue45057 first they will be much simpler.

Issue29152 can be related. If call addError() and addFailure() from addSubTest(), PR 28082 should be rewritten.
History
Date User Action Args
2021-08-31 08:16:13serhiy.storchakasetrecipients: + serhiy.storchaka, pitrou, rbcollins, ezio.melotti, r.david.murray, michael.foord, martin.panter, zach.ware
2021-08-31 08:16:13serhiy.storchakasetmessageid: <1630397773.4.0.568148134648.issue25894@roundup.psfhosted.org>
2021-08-31 08:16:13serhiy.storchakalinkissue25894 messages
2021-08-31 08:16:13serhiy.storchakacreate