Message180233
Nice/elegant idea. A couple comments:
(1) What will be the semantics of TestCase/subtest failures? Currently, it looks like each subtest failure registers as an additional failure, meaning that the number of test failures can exceed the number of test cases. For example (with a single test case with 2 subtests):
$ ./python.exe test_subtest.py
FF
======================================================================
FAIL: test (__main__.MyTests) (i=0)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subtest.py", line 9, in test
self.assertEqual(0, 1)
AssertionError: 0 != 1
======================================================================
FAIL: test (__main__.MyTests) (i=1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subtest.py", line 9, in test
self.assertEqual(0, 1)
AssertionError: 0 != 1
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=2)
With the way I understand it, it seems like a subtest failure should register as a failure of the TestCase as a whole, unless the subtests should be enumerated and considered tests in their own right (in which case the total test count should reflect this).
(2) Related to (1), it doesn't seem like decorators like expectedFailure are being handled correctly. For example:
@unittest.expectedFailure
def test(self):
for i in range(2):
with self.subTest(i=i):
self.assertEqual(i, 0)
This results in:
Ran 1 test in 0.001s
FAILED (failures=1, unexpected successes=1)
In other words, it seems like the decorator is being applied to each subtest as opposed to the test case as a whole (though actually, I think the first should read "expected failures=1"). It seems like one subtest failing should qualify as an expected failure, or are the semantics such that expectedFailure means that every subtest must fail? |
|
Date |
User |
Action |
Args |
2013-01-19 00:33:30 | chris.jerdonek | set | recipients:
+ chris.jerdonek, spiv, exarkun, ncoghlan, pitrou, ezio.melotti, eric.araujo, r.david.murray, michael.foord, brian.curtin, hpk, fperez, Yaroslav.Halchenko, santoso.wijaya, nchauvat, kynan, Julian, abingham, eric.snow, serhiy.storchaka, borja.ruiz, bfroehle |
2013-01-19 00:33:30 | chris.jerdonek | set | messageid: <1358555610.83.0.527134617854.issue16997@psf.upfronthosting.co.za> |
2013-01-19 00:33:30 | chris.jerdonek | link | issue16997 messages |
2013-01-19 00:33:30 | chris.jerdonek | create | |
|