Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail-fast behavior for unittest #52321

Closed
jcd mannequin opened this issue Mar 5, 2010 · 6 comments
Closed

Fail-fast behavior for unittest #52321

jcd mannequin opened this issue Mar 5, 2010 · 6 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@jcd
Copy link
Mannequin

jcd mannequin commented Mar 5, 2010

BPO 8074
Nosy @amauryfa, @voidspace

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/voidspace'
closed_at = <Date 2010-03-27.17:37:33.282>
created_at = <Date 2010-03-05.17:32:22.708>
labels = ['type-feature', 'library']
title = 'Fail-fast behavior for unittest'
updated_at = <Date 2010-03-27.17:37:33.280>
user = 'https://bugs.python.org/jcd'

bugs.python.org fields:

activity = <Date 2010-03-27.17:37:33.280>
actor = 'michael.foord'
assignee = 'michael.foord'
closed = True
closed_date = <Date 2010-03-27.17:37:33.282>
closer = 'michael.foord'
components = ['Library (Lib)']
creation = <Date 2010-03-05.17:32:22.708>
creator = 'jcd'
dependencies = []
files = []
hgrepos = []
issue_num = 8074
keywords = []
message_count = 6.0
messages = ['100493', '100494', '100792', '100799', '101473', '101832']
nosy_count = 3.0
nosy_names = ['amaury.forgeotdarc', 'jcd', 'michael.foord']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue8074'
versions = ['Python 2.7', 'Python 3.2']

@jcd
Copy link
Mannequin Author

jcd mannequin commented Mar 5, 2010

When a long suite of tests is running, it would be nice to be able to exit out early to see the results of the tests that have run so far.

I would like to see a couple of features included in unittest to this effect that have recently been implemented in django's test runner. The first exits gracefully on ^C, the second allows the user to specify a --failfast option, which causes the test runner to exit after the first failure or error, rather than proceeding with the

If you hit ^C, rather than exit with a stack trace of whatever was happening at the time, unittest should complete the current test, and then exit with the results to that point. A second ^C would be treated as a normal KeyboardInterrupt, in case that last test were somehow running on too long.

{{{
$ python test_module.py
..F.s.^C <Test run halted by Ctrl-C> x
======================================================================
FAIL: test_multiline_equal (main.MyTest)
Unicode objects return diffs on failure
----------------------------------------------------------------------

Traceback (most recent call last):
  File "test_test.py", line 39, in test_multiline_equal
    self.assertEqual(original, revised)
AssertionError: 
- Lorem ipsum dolor sit amet, 
?             ^
+ Lorem ipsum color sit amet, 
?             ^
              consectetur adipisicing elit, 
+             that's the way the cookie crumbles,
              sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Ran 7 tests in 12.010s

FAILED (failures=1, skipped=1, expected failures=1)
}}}

A failfast option could be specified as a keyword argument to unittest.main(), or unittest.main() could read it off the command line, according to the same mechanism used by, e.g., the verbosity option.

@jcd jcd mannequin added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Mar 5, 2010
@jcd
Copy link
Mannequin Author

jcd mannequin commented Mar 5, 2010

This feature was implemented in the django test runner as r12034 (^C override) and r11843 (failfast option)

@amauryfa
Copy link
Member

fwiw, the similar bpo-2241 was rejected two years ago.

@voidspace
Copy link
Contributor

I like the feature. I'm a bit concerned about the proliferation of command line options for unittest though.

Steve's rejection of the feature seemed to be on the basis that he didn't need it (or see a need for it). :-)

@voidspace
Copy link
Contributor

Failfast option added, committed revision 79265. Still needs documentation. Ctrl-C handling not yet done. Slightly more complex as it needs to play well with tests that test SIGINT handling.

@voidspace
Copy link
Contributor

Ctrl-C handling now implemented (-c command line option). Still needs documentation, but there is a bunch of this outstanding and I will do it in one go and rework the unittest docs at the same time.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants