classification
Title: Fail-fast behavior for unittest
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: michael.foord Nosy List: amaury.forgeotdarc, jcd, michael.foord
Priority: normal Keywords:

Created on 2010-03-05 17:32 by jcd, last changed 2010-03-27 17:37 by michael.foord. This issue is now closed.

Messages (6)
msg100493 - (view) Author: Cliff Dyer (jcd) Date: 2010-03-05 17:32
When a long suite of tests is running, it would be nice to be able to exit out early to see the results of the tests that have run so far.  

I would like to see a couple of features included in `unittest` to this effect that have recently been implemented in django's test runner.  The first exits gracefully on `^C`, the second allows the user to specify a `--failfast` option, which causes the test runner to exit after the first failure or error, rather than proceeding with the 

If you hit `^C`, rather than exit with a stack trace of whatever was happening at the time, `unittest` should complete the current test, and then exit with the results to that point.  A second `^C` would be treated as a normal `KeyboardInterrupt`, in case that last test were somehow running on too long.

{{{
  $ python test_module.py
..F.s.^C <Test run halted by Ctrl-C> x
======================================================================
FAIL: test_multiline_equal (__main__.MyTest)
Unicode objects return diffs on failure
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_test.py", line 39, in test_multiline_equal
    self.assertEqual(original, revised)
AssertionError: 
- Lorem ipsum dolor sit amet, 
?             ^
+ Lorem ipsum color sit amet, 
?             ^
              consectetur adipisicing elit, 
+             that's the way the cookie crumbles,
              sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

----------------------------------------------------------------------
Ran 7 tests in 12.010s

FAILED (failures=1, skipped=1, expected failures=1)
}}}

A `failfast` option could be specified as a keyword argument to `unittest.main()`, or `unittest.main()` could read it off the command line, according to the same mechanism used by, e.g., the `verbosity` option.
msg100494 - (view) Author: Cliff Dyer (jcd) Date: 2010-03-05 17:37
This feature was implemented in the django test runner as [r12034](http://code.djangoproject.com/changeset/12034) (^C override) and [r11843](http://code.djangoproject.com/changeset/11843) (failfast option)
msg100792 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-03-10 16:21
fwiw, the similar issue2241 was rejected two years ago.
msg100799 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-03-10 16:47
I like the feature. I'm a bit concerned about the proliferation of command line options for unittest though. 

Steve's rejection of the feature seemed to be on the basis that he didn't need it (or see a need for it). :-)
msg101473 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-03-22 01:04
Failfast option added, committed revision 79265. Still needs documentation. Ctrl-C handling not yet done. Slightly more complex as it needs to play well with tests that test SIGINT handling.
msg101832 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-03-27 17:37
Ctrl-C handling now implemented (-c command line option). Still needs documentation, but there is a bunch of this outstanding and I will do it in one go and rework the unittest docs at the same time.
History
Date User Action Args
2010-03-27 17:37:33michael.foordsetstatus: open -> closed

messages: + msg101832
stage: needs patch -> resolved
2010-03-22 01:04:22michael.foordsetresolution: accepted
messages: + msg101473
2010-03-10 16:47:13michael.foordsetmessages: + msg100799
2010-03-10 16:21:47amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg100792
2010-03-05 17:43:49brian.curtinsetversions: - Python 3.1, Python 3.3
nosy: + michael.foord

priority: normal
assignee: michael.foord
stage: needs patch
2010-03-05 17:37:29jcdsetmessages: + msg100494
2010-03-05 17:32:22jcdcreate