Author p-ganssle
Recipients ezio.melotti, michael.foord, p-ganssle, rbcollins
Date 2018-11-27.15:24:27
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1543332268.01.0.788709270274.issue35327@psf.upfronthosting.co.za>
In-reply-to
Content
It seems that if you call skipTest *anywhere* in a test function, even in a subTest, the *entire* function gets marked as "skipped", even if only one sub-test is skipped.

Example:

    import unittest

    class SomeTest(unittest.TestCase):
        def test_something(self):
            for i in range(1, 3):
                with self.subTest(i):
                    if i > 1:
                        self.skipTest('Not supported')

                    self.assertEqual(i, 1)

If you run `python3.7 -m unittest -v` on this, you get:

    $ python -m unittest -v
    test_something (test_mod.SomeTest) ... skipped 'Not supported'

    ----------------------------------------------------------------------
    Ran 1 test in 0.000s

    OK (skipped=1)


Despite the fact that the test *was* run in the `i == 1` case. Similarly, pytest marks this as a single skipped test:

    ========= test session starts =======
    
    platform linux -- Python 3.7.1, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
    rootdir: /tmp/test_mod, inifile:
    collected 1 item
    
    test_mod.py s                                           [100%]
    
    ===== 1 skipped in 0.00 seconds =====


The solution to this is not obvious, unfortunately. One way to solve it would be to have each subtest considered a "separate test". Another would be to detect whether subTests have been skipped and expose only the *skipped* tests as separate tests. You could also add a "partially skipped" marker, so to say, "Some parts of this were skipped"

I suspect the right answer is some combination of these three answers - possibly adding an extra-verbose mode that splits out all sub tests and having skipped subTests default to being displayed separately.

Somewhat related to issue #30997.
History
Date User Action Args
2018-11-27 15:24:28p-gansslesetrecipients: + p-ganssle, rbcollins, ezio.melotti, michael.foord
2018-11-27 15:24:28p-gansslesetmessageid: <1543332268.01.0.788709270274.issue35327@psf.upfronthosting.co.za>
2018-11-27 15:24:27p-gansslelinkissue35327 messages
2018-11-27 15:24:27p-gansslecreate