classification
Title: sys.exit() in a test causes a test run to die
Type: behavior Stage: resolved
Components: Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: michael.foord Nosy List: eric.araujo, michael.foord, rbcollins, ysj.ray
Priority: normal Keywords: easy, patch

Created on 2010-12-03 01:25 by michael.foord, last changed 2014-11-07 11:15 by michael.foord. This issue is now closed.

Files
File name Uploaded Description Edit
issue_10611.diff ysj.ray, 2010-12-10 06:29 patch against py3k
Messages (9)
msg123153 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-12-03 01:25
Reported by a unittest2 user. 

A SystemExit (or GeneratorExit) will cause a test run to stop in 2.7 / 3.2. This would just be reported as an error in 2.6.

>>> from unittest import TestCase
>>> def test(s):
...  raise GeneratorExit
... 
>>> class T(TestCase):
...  test = test
... 
>>> t = T('test')
>>> t.run()


Above code works in Python 2.6 (the exception is caught by TestCase.run) but dies in 2.7 / 3.2
msg123717 - (view) Author: ysj.ray (ysj.ray) Date: 2010-12-10 06:29
Agreed. I think the "except Exception" in TestCase.run() should be "except BaseException", since BaseException could catch Exception, SystemExit, GeneratorExit, KeyboardInterrupt. The KeyboardInterrupt should be caught first. The remaining three is exactly what is needed.

Here is a patch I worked, with unittest.
msg123718 - (view) Author: √Čric Araujo (eric.araujo) * (Python committer) Date: 2010-12-10 07:15
LGTM.
msg123726 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-12-10 10:43
At the moment exception handling for setUp / tearDown / testMethod and cleanUp functions are all handled separately. They all have to call addError and as a result we have inconsistent handling of skips, expected failures (etc). There are separate issues for handling expected failures in setUp and skips in tearDown.

I'd like to fix all these issues by moving the exception handling into a single method and unifying the reporting of failure / error / expected failure / skip test. This will fix all these issues and nicely simplify the implementation.
msg124110 - (view) Author: ysj.ray (ysj.ray) Date: 2010-12-16 07:38
> I'd like to fix all these issues by moving the exception handling into a single method and unifying the reporting of failure / error / expected failure / skip test. This will fix all these issues and nicely simplify the implementation.

That sounds good.
msg124356 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-12-19 14:56
Committed to Python 2.7 in revision 87406.
msg124357 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2010-12-19 15:00
Committed to py3k in revision 87390.
msg230788 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2014-11-07 10:37
Hmm, so testtools went in a different direction here - the same unification stuff, but see https://github.com/testing-cabal/testtools/commit/18bc5741cf277f7a0d601568be6dccacc7b0783c

tl;dr - I think unittest should not prevent this causing the process to exit (but it should still fail the test and fail the test run as a whole), analgous to how KeyboardInterrupt is handled.
msg230790 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-11-07 11:15
Allowing sys.exit() to end the test run was particularly a problem for testing command line tools, where improper patching / unexpected code paths would trigger a sys.exit.

If a test framework author wants a way to end the test run I'm happy to provide that (a custom exception to raise or a flag to turn off sys.exit handling), but I don't think having sys.exit kill test runs is best for test authors.
History
Date User Action Args
2014-11-07 11:15:15michael.foordsetmessages: + msg230790
2014-11-07 10:37:20rbcollinssetnosy: + rbcollins
messages: + msg230788
2010-12-19 15:00:23michael.foordsetnosy: eric.araujo, michael.foord, ysj.ray
messages: + msg124357
2010-12-19 14:56:28michael.foordsetstatus: open -> closed
nosy: eric.araujo, michael.foord, ysj.ray
messages: + msg124356

resolution: fixed
stage: patch review -> resolved
2010-12-16 07:38:25ysj.raysetnosy: eric.araujo, michael.foord, ysj.ray
messages: + msg124110
2010-12-10 10:43:09michael.foordsetmessages: + msg123726
2010-12-10 07:15:58eric.araujosetmessages: + msg123718
stage: test needed -> patch review
2010-12-10 06:29:03ysj.raysetfiles: + issue_10611.diff

nosy: + ysj.ray
messages: + msg123717

keywords: + patch
2010-12-03 01:34:17eric.araujosetnosy: + eric.araujo
2010-12-03 01:25:34michael.foordsettitle: sys.exit() in a test causes the run to stp -> sys.exit() in a test causes a test run to die
2010-12-03 01:25:17michael.foordcreate