classification
Title: Change unittest's _SubTest to not sort its params when printing test failures
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eric.smith, louielu, nitishch, serhiy.storchaka
Priority: normal Keywords: easy

Created on 2017-06-14 12:00 by eric.smith, last changed 2017-06-18 06:01 by serhiy.storchaka.

Pull Requests
URL Status Linked Edit
PR 2265 open serhiy.storchaka, 2017-06-18 05:31
Messages (8)
msg295998 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2017-06-14 12:00
Now that **kwargs are sorted, it would be better if the error given by a unittest subTest (as implemented in uniitest._case._SubTest) didn't sort the parameters when they're printed, but instead printed them out in order.

This might be complicated by the ChainMap that's used as part of the implementation, but it should still be doable.

For example, I have code that has:

    with self.subTest(hash=hash, cmp=cmp, frozen=frozen):

But when it errors out, it produces:

    FAIL: test_hash_rules (tst.TestCase) (cmp=True, frozen=True, hash=None)

It would be easier to check my code if the order the values was printed was the same as the order in the self.subTest() call.
msg296001 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2017-06-14 12:34
Correction: it's implemented in unittest.case._SubTest, specifically in  _subDescription().
msg296002 - (view) Author: Louie Lu (louielu) * Date: 2017-06-14 12:41
I think the question will be, when using multiple subTest (this is why using ChainMap I think), how to determine their order?:

    with self.subTest(c=i, b=i, a=i):
        with self.subTest(b=i, c=50, a=60):
	    self.assertEqual(i, 2)


>>> FAIL: test_foo (__main__.Test) (a=60, b=4, c=50)
msg296003 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2017-06-14 13:17
Good question.

It looks like ChainMap does something I wouldn't expect:

>>> for k, v in ChainMap({'a': 0, 'b': 1, 'c': 2}, {'b': 3, 'a': 4}).items():
...  print(k, v)
... 
b 1
a 0
c 2

Once we define what we'd like the output to look like, I'm sure it would be easy enough to get the order right.
msg296004 - (view) Author: Louie Lu (louielu) * Date: 2017-06-14 13:21
Additional note, the order will changed it random way in ChainMap, e.g.:


>>> for k, v in ChainMap({'a': 0, 'b': 1, 'c': 2}, {'b': 3, 'a': 4}).items(): print(k, v)
...
a 0
c 2
b 1

-----restart----

>>> for k, v in ChainMap({'a': 0, 'b': 1, 'c': 2}, {'b': 3, 'a': 4}).items(): print(k, v)
...
b 1
c 2
a 0
msg296005 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2017-06-14 13:26
Correct on the order changed with regular dicts. That's why I'm targeting this specifically for Python 3.7 and with **kwargs, where order is guaranteed. We might have to use a structure other than a ChainMap of dicts, like a ChainMap of OrderDicts.
msg296261 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-18 05:34
PR 2265 implements a private subclass of ChainMap that preserves ordering.
msg296263 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-18 06:01
Note that the order of parameters of nested subtests is from inner to outer.
History
Date User Action Args
2017-06-18 06:01:21serhiy.storchakasetmessages: + msg296263
2017-06-18 05:34:13serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg296261
2017-06-18 05:31:49serhiy.storchakasetpull_requests: + pull_request2316
2017-06-14 13:26:27eric.smithsetmessages: + msg296005
2017-06-14 13:21:21louielusetmessages: + msg296004
2017-06-14 13:17:12eric.smithsetmessages: + msg296003
2017-06-14 13:12:58nitishchsetnosy: + nitishch
2017-06-14 12:41:34louielusetnosy: + louielu
messages: + msg296002
2017-06-14 12:34:30eric.smithsetmessages: + msg296001
title: Change unittest's _SubTest to not sort its params -> Change unittest's _SubTest to not sort its params when printing test failures
2017-06-14 12:00:09eric.smithcreate