classification
Title: Confusing output for TestCase.subTest(0)
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Nan Wu, berker.peksag, ezio.melotti, michael.foord, pitrou, python-dev, rbcollins, zach.ware
Priority: normal Keywords: easy, patch

Created on 2015-11-18 02:34 by ezio.melotti, last changed 2017-03-31 16:36 by dstufft. This issue is now closed.

Files
File name Uploaded Description Edit
subtest_msg_check_against_None.patch Nan Wu, 2015-11-21 17:12 review
Pull Requests
URL Status Linked Edit
PR 552 closed dstufft, 2017-03-31 16:36
Messages (5)
msg254827 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2015-11-18 02:34
When a single positional argument is passed to subTest(), if the argument is false, its value won't be displayed in the output -- (<subtest>) will appear instead:

>>> import unittest
>>> class NumbersTest(unittest.TestCase):
...   def test_odd(self):
...     for i in range(4):
...       with self.subTest(i):  # single positional arg
...         self.assertNotEqual(i%2, 0)
... 
>>> unittest.main(exit=False)
======================================================================
FAIL: test_odd (__main__.NumbersTest) (<subtest>)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<stdin>", line 5, in test_odd
AssertionError: 0 == 0
======================================================================
FAIL: test_odd (__main__.NumbersTest) [2]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<stdin>", line 5, in test_odd
AssertionError: 0 == 0
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=2)

This is because subTest() accepts a positional "msg" arg, passes it to _SubTest (Lib/unittest/case.py:515), and then _SubTest checks using "if self._message:" (Lib/unittest/case.py:1400).

I think it would be better to check the message against a sentinel value instead.
msg255064 - (view) Author: Nan Wu (Nan Wu) * Date: 2015-11-21 17:12
Made it check against None explicitly. My concern is if [] is passed in, if will show [[]]. But this case should be rare.
msg256574 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2015-12-17 04:37
I think Ezio's suggestion of a sentinel value would be better, allowing None to be using as a legitimate 'message' [1].  That is, somewhere at global scope, define '_subtest_msg_sentinel = object()', change the msg default at Lib/unittest/case.py:500 to be 'msg=_subtest_msg_sentinel', and change the check at Lib/unittest/case.py:1400 to check 'if message is not _subtest_msg_sentinel'.


[1] For example:

   class TruthTest(unittest.TestCase):
       def test_truth(self):
           for o in None, 1, 0, [], (4,):
               with self.subTest(o):
                   self.assertTrue(o)

Should print failure results including '[None]', '[0]', and '[[]]'.
msg277170 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-21 16:34
New changeset ba743894e793 by Berker Peksag in branch '3.5':
Issue #25651: Allow falsy values to be used for msg parameter of subTest()
https://hg.python.org/cpython/rev/ba743894e793

New changeset ddbf92168a44 by Berker Peksag in branch '3.6':
Issue #25651: Merge from 3.5
https://hg.python.org/cpython/rev/ddbf92168a44

New changeset e5888f5b9cf8 by Berker Peksag in branch 'default':
Issue #25651: Merge from 3.6
https://hg.python.org/cpython/rev/e5888f5b9cf8
msg277171 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-09-21 16:36
Fixed. I lost some time because of this today :)
History
Date User Action Args
2017-03-31 16:36:35dstufftsetpull_requests: + pull_request1077
2016-09-21 16:36:57berker.peksagsetstatus: open -> closed

versions: + Python 3.7, - Python 3.4
nosy: + berker.peksag

messages: + msg277171
resolution: fixed
stage: test needed -> resolved
2016-09-21 16:34:40python-devsetnosy: + python-dev
messages: + msg277170
2015-12-17 04:37:01zach.waresetnosy: + zach.ware
messages: + msg256574
2015-11-21 17:12:46Nan Wusetfiles: + subtest_msg_check_against_None.patch

nosy: + Nan Wu
messages: + msg255064

keywords: + patch
2015-11-18 02:34:13ezio.melotticreate