Author ncoghlan
Recipients Julian, Yaroslav.Halchenko, abingham, bfroehle, borja.ruiz, brian.curtin, chris.jerdonek, eric.araujo, eric.snow, exarkun, ezio.melotti, fperez, hpk, kynan, michael.foord, nchauvat, ncoghlan, pitrou, r.david.murray, santoso.wijaya, serhiy.storchaka, spiv
Date 2013-01-19.14:49:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1358606971.41.0.253488148199.issue16997@psf.upfronthosting.co.za>
In-reply-to
Content
I think we're going to have to separate out two counts in the metrics - the total number of tests (the current counts), and the total number of subtests (the executed subtest blocks). (Other parameterisation solutions can then choose whether to treat each pair of parameters as a distinct test case or as a subtest - historical solutions would appear as distinct test cases, while new approaches might choose to use the subtest machinery).

The aggregation of subtest results to test case results would then be that the test case fails if either:
- an assertion directly in the test case fails
- an assertion fails in at least one subtest of that test case

The interpretation of "expected failure" in a world with subtests is then clear: as long as at least one subtest or assertion fails, the decorator is satisfied that the expected test case failure is occurred.
History
Date User Action Args
2013-01-19 14:49:31ncoghlansetrecipients: + ncoghlan, spiv, exarkun, pitrou, ezio.melotti, eric.araujo, r.david.murray, michael.foord, brian.curtin, hpk, fperez, chris.jerdonek, Yaroslav.Halchenko, santoso.wijaya, nchauvat, kynan, Julian, abingham, eric.snow, serhiy.storchaka, borja.ruiz, bfroehle
2013-01-19 14:49:31ncoghlansetmessageid: <1358606971.41.0.253488148199.issue16997@psf.upfronthosting.co.za>
2013-01-19 14:49:31ncoghlanlinkissue16997 messages
2013-01-19 14:49:30ncoghlancreate