classification
Title: Clarify unittest expectedFailure behaviour in the documentation
Type: behavior Stage: patch review
Components: Documentation, Tests Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Kit Choi2, docs@python, iritkatriel, miss-islington, steve.dower
Priority: normal Keywords: patch

Created on 2019-09-30 09:14 by Kit Choi2, last changed 2020-10-19 21:28 by miss-islington.

Pull Requests
URL Status Linked Edit
PR 22740 merged iritkatriel, 2020-10-17 14:52
PR 22782 open miss-islington, 2020-10-19 21:27
PR 22783 open miss-islington, 2020-10-19 21:28
Messages (2)
msg353557 - (view) Author: Kit Choi (Kit Choi2) Date: 2019-09-30 09:14
Following discussion in https://bugs.python.org/issue38296

The docstring of unittest.expectedFailure is misleading for people who differentiate "error" and "failure" when they read the sentence. This has a consequence of developers using the decorator without noticing unexpected errors are also silenced, which mean running a risk of their tests becoming unmaintained and invalid in the future.

I suggest updating the documentation to include a mention of the current behaviour of silencing unexpected errors, so that developers are aware of this.

Something like this?:

Mark the test such that unexpected success results in a failure. If an exception (BaseException excluding KeyboardInterrupt) occurs, the test will be considered a success. If the test passes, it will be considered a failure.
msg379023 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-10-19 21:27
New changeset fa8748271a61177e9bf609921fa464cc6990478b by Irit Katriel in branch 'master':
bpo-38320: Clarify that expectedFailure is satisfied by either failure or error of the test. (GH-22740)
https://github.com/python/cpython/commit/fa8748271a61177e9bf609921fa464cc6990478b
History
Date User Action Args
2020-10-19 21:28:13miss-islingtonsetpull_requests: + pull_request21742
2020-10-19 21:27:35miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request21741
2020-10-19 21:27:25steve.dowersetnosy: + steve.dower
messages: + msg379023
2020-10-17 14:52:57iritkatrielsetkeywords: + patch
nosy: + iritkatriel

pull_requests: + pull_request21703
stage: patch review
2019-09-30 09:14:01Kit Choi2create