classification
Title: Change in expectedFailure breaks testtools
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, barry, cheryl.sabella, ezio.melotti, larry, michael.foord, pitrou, rbcollins, thomir
Priority: normal Keywords:

Created on 2014-02-19 18:20 by barry, last changed 2019-02-21 02:45 by cheryl.sabella. This issue is now closed.

Files
File name Uploaded Description Edit
test_expected_failure.py barry, 2014-02-19 18:20
Messages (8)
msg211636 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2014-02-19 18:20
The change for issue #16997 82844:5c09e1c57200 introduced a regression with testtools.  See attached file and instructions for reproducing it (it requires the testtools package).
msg211639 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2014-02-19 18:35
I concur that this is a regression - "     

unittest.expectedFailure()

    Mark the test as an expected failure. If the test fails when run, the test is not counted as a failure.
"

is in the public docs for the unittest module, and depending on a private attribute to be set is a bit of an abstraction violation - at the least it should handle it not being set (for any older test unittest compatible testcase object).
msg211640 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2014-02-19 18:36
Bumping to release blocker so Larry can make a determination.
msg211689 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-02-20 05:49
So here's the thing.  It only works in Python 3.3 because testtools depends on internal implementation details of unittest, specifically a private exception ("unittest.case._UnexpectedSuccess").  The public interface still works fine.  So I don't think this is a regression.  If testtools is willing to release hacks depending on undocumented internal behavior, I don't think it's unreasonable for them to rework their hack to support Python 3.4.

That said, it might be nice to not break testtools if we can avoid it.  Antoine: would it be reasonable to rework the implementation of subTest in a way that permitted us to revert the changes to expectedFailure?
msg211690 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2014-02-20 05:52
Oh! I didn't realise it was due to us looking at a private exception - I haven't been given a traceback to review, just the statement of a problem.... We shouldn't have done that(and *Definitely* should have filed a bug that we needed to).

So - I think the question should become 'can we cope with this in testtools' - we'll issue a point release to ensure compat, assuming it's at all possible.
msg211718 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-20 10:54
> Antoine: would it be reasonable to rework the implementation of
> subTest in a way that permitted us to revert the changes to
> expectedFailure?

I can't tell you for sure, but certainly not in an easy way. Expected failures were the most delicate feature to keep compatible when adding subtests.
(in other words: not for 3.4, IMO)
msg211748 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-02-20 20:01
Then I'm marking this as 3.5 and clearing the regression keyword.  From here we can either fix it in 3.5 or not bother (and just close this issue as wontfix).
msg336177 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2019-02-21 02:45
Larry suggested closing this as wontfix 5 years ago.  Since testtools added code for this test and there hasn't been any other changes since, I'll close per his suggestion.
History
Date User Action Args
2019-02-21 02:45:08cheryl.sabellasetstatus: open -> closed

nosy: + cheryl.sabella
messages: + msg336177

resolution: wont fix
stage: resolved
2014-02-20 20:01:15larrysetkeywords: - 3.4regression
priority: high -> normal
messages: + msg211748

versions: + Python 3.5, - Python 3.4
2014-02-20 10:54:55pitrousetmessages: + msg211718
2014-02-20 05:52:28rbcollinssetmessages: + msg211690
2014-02-20 05:50:10larrysetpriority: release blocker -> high
2014-02-20 05:49:57larrysetmessages: + msg211689
2014-02-20 02:54:14Arfreversetnosy: + Arfrever
2014-02-19 19:00:53ezio.melottisetnosy: + pitrou, ezio.melotti, michael.foord
type: behavior
2014-02-19 18:36:55barrysetpriority: normal -> release blocker
nosy: + larry
messages: + msg211640

2014-02-19 18:35:41rbcollinssetmessages: + msg211639
2014-02-19 18:34:13barrysetnosy: + rbcollins
2014-02-19 18:24:22thomirsetnosy: + thomir
2014-02-19 18:20:23barrycreate