Author chris.jerdonek
Recipients Arfrever, Julian, Yaroslav.Halchenko, abingham, bfroehle, borja.ruiz, brett.cannon, brian.curtin, chris.jerdonek, eric.araujo, eric.snow, exarkun, ezio.melotti, fperez, hpk, michael.foord, nchauvat, ncoghlan, pitrou, r.david.murray, santoso.wijaya, serhiy.storchaka, spiv
Date 2013-01-29.01:47:56
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1359424078.82.0.423308773055.issue16997@psf.upfronthosting.co.za>
In-reply-to
Content
>> 1. Easily append data to failure messages coming from a block of asserts
>> 2. Continue running a test case after a failure from a block of asserts
>
>> Both of these seem independently useful and more generally applicable,
>
> I don't understand what you mean. 1 is pointless without 2 (if you let the exception bubble up, unittest already deals with it). 2 without 1 doesn't make much sense either (if you only want to silence an exception, a simple try...except will suffice).

I'll explain.  (1) is useful without (2) because it lets you add information to the failure data for a group of asserts without having to use the "msg" parameter in every call to assert().  This is useful, for example, if you're testing a number of cases in a loop (with the current behavior of ending the test on first failure), and it's not clear from the default exception message which iteration of the loop failed.  Your original example is such a case (minus the part about continuing in case of failure).  This use case is basically the one addressed by Serhiy's suggestion in this message:

http://bugs.python.org/issue16997#msg180225

(2) is useful without (1) if you'd like to get information about more than one assertion failure in a TestCase (just as in your proposal), but the assertions aren't necessarily coming from a "parametrization" or different iterations of a loop.  With the proposed API, you'd do something like:

    with self.subTest():
        # First assertion
        ...
    with self.subTest():
        # Second assertion
        ...

The difference here is that I wouldn't call these "subtests," and you don't need parameter data to know which assertion is at fault.  The meaning here is more like "with self.continueTesting()".

> Also, changing the exception message might be confusing as readers expect the displayed message to be exactly str(exc).

If the API is more like self.assert*()'s "msg" parameter which appends data to the usual exception, then it will be the same as what people are already used to.  Also see the "longMessage" attribute (which defaults to True), which separates the extra message data from the default exception message:

http://docs.python.org/dev/library/unittest.html#unittest.TestCase.longMessage

>> Alternatively, subtests could be made independently addressable and
>> runnable, but that route seems more challenging and less certain.
>
> I would say impossible, unless you know a way to run a `with` block in isolation ;-)

I'm not advocating independent addressability/runnability of subtests or the following approach, but a naive way to do this would be to run the TestCase as usual, but skip over any subTest blocks if the parameter data isn't an exact match.
History
Date User Action Args
2013-01-29 01:47:59chris.jerdoneksetrecipients: + chris.jerdonek, brett.cannon, spiv, exarkun, ncoghlan, pitrou, ezio.melotti, eric.araujo, Arfrever, r.david.murray, michael.foord, brian.curtin, hpk, fperez, Yaroslav.Halchenko, santoso.wijaya, nchauvat, Julian, abingham, eric.snow, serhiy.storchaka, borja.ruiz, bfroehle
2013-01-29 01:47:58chris.jerdoneksetmessageid: <1359424078.82.0.423308773055.issue16997@psf.upfronthosting.co.za>
2013-01-29 01:47:58chris.jerdoneklinkissue16997 messages
2013-01-29 01:47:56chris.jerdonekcreate