classification
Title: decouple unittest assertions from the TestCase class
Type: enhancement Stage:
Components: Tests Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Gregory.Salvan, Julian, martin.panter, merwok, michael.foord, ncoghlan, r.david.murray, rbcollins, serhiy.storchaka
Priority: normal Keywords:

Created on 2013-11-18 14:38 by Gregory.Salvan, last changed 2015-11-10 02:38 by martin.panter.

Messages (6)
msg203295 - (view) Author: Gregory Salvan (Gregory.Salvan) Date: 2013-11-18 14:38
Actually unittest assertions depends on testcase class forcing us to extend it to add assertions and to use it to make assertions outside tests.

Seeing interests in rethinking the way assertions are done in unittest, this issue first intent to collect feedback in order to suggest an implementation that fit the most.

Some notes from private discussions:
- it was briefly discussed here #18054.
- taking care of popular solutions like py.test's rich assert
statements and the testtools matcher objects.
- avoid unnecessary complexity, staying focused on value

My opinion:
- must not change unittest api
- may be put in a seperate package (splitting "unittest" in "unittest" and "assertions")
- Open to hide assertions exceptions in optimized mode or providing a simple way to change default behaviour (log, skip... instead of throwing unhandled exception).
msg203303 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-11-18 15:44
You should probably start by summarizing the assertThat proposal from issue 18054, which suggested making a separate issue for assertThat.
msg203351 - (view) Author: Gregory Salvan (Gregory.Salvan) Date: 2013-11-19 10:30
issue18054 :

- adding assertCleanError in the ipaddress module, 
- suggesting assertCleanTraceback, assertRaisedFrom in unittest 

-> usefull but TestCase has already a wide api.

A solution like Testtools assertThat with matcher protocol (https://testtools.readthedocs.org/en/latest/for-test-authors.html#matchers) would not expand too much TestCase api and permit to easily extend assertions.
msg227588 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-26 07:13
Here are most popular idioms which deserve special assertion methods:

assertHasAttr(obj, name) == assertTrue(hasattr(obj, name))
assertIsSubclass(type, expected) == assertTrue(issubclass(type, expected))
assertTypeIs(obj, expected) == assertIs(type(obj), expected)
assertTypedEqual(actual, expected) == assertIs(type(actual), type(expected)) and assertEqual(actual, expected) # or assertIsInstance(actual, type(expected))?
assertStartsWith(actual, prefix) == assertTrue(s.startswith(prefix))
assertEndsWith(actual, suffix) == assertTrue(s.endswith(suffix))
assertUnorderedSequenceEqual(first, second) == assertTrue(all(x in second for x in first)) and assertTrue(all(x in first for x in second)) and assertEqual(len(first), len(second))
msg230813 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2014-11-07 15:27
Hi, I'm glad you're interested in this. I very much want to see a matcher/hamcrest approach rather than a library of assertions per se - because match-or-except makes layering things harder.
msg254427 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-10 02:38
Looking at Gregory and Robert’s “matchers” link, which also covers “assertThat”, it seems to provide a library of matcher assertion objects. E.g. instead of this TestCase-coupled code:

self.assertEquals(something, "expected value")

you would write

from testtools.matchers import Equals
self.assertThat(something, Equals("expected value"))

Implementing a custom matcher (say HasAttr for Serhiy’s first use case) seems to involve creating the HasAttr class with a match() method, and maybe another HasAttrMismatch class (though maybe you could get away with just a shared generic mismatch class?).

It seems to me that if you ignore the vaguely-documented TestCase.failureException attribute, you can fairly easily decouple the existing methods from a TestCase instance:

def assert_equal(first, second, msg=None):
    # Implementation could be copied independently of any test state
    TestCase().assertEqual(first, second, msg)

The matcher scheme would be more flexible though, because you can compose matcher objects.
History
Date User Action Args
2015-11-10 02:38:20martin.pantersetmessages: + msg254427
2014-11-07 15:27:53rbcollinssetmessages: + msg230813
2014-09-26 07:13:28serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg227588
2014-08-06 02:49:54martin.pantersetnosy: + martin.panter
2013-11-29 16:57:46Juliansetnosy: + Julian
2013-11-22 20:51:58merwoksetnosy: + merwok
2013-11-19 10:30:42Gregory.Salvansetmessages: + msg203351
2013-11-18 15:44:02r.david.murraysetnosy: + r.david.murray

messages: + msg203303
title: Improving unittest assertions -> decouple unittest assertions from the TestCase class
2013-11-18 14:38:14Gregory.Salvancreate