Title: Exceptions and arguments disappear when using argparse inside with statement
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4, Python 3.5, Python 2.7
Status: closed Resolution: duplicate
Dependencies: 9938 Superseder:
Assigned To: Nosy List: benjamin.peterson, invisibleroads, r.david.murray
Priority: normal Keywords:

Created on 2015-04-28 17:57 by invisibleroads, last changed 2015-05-06 17:52 by r.david.murray. This issue is now closed.

File name Uploaded Description Edit invisibleroads, 2015-04-28 17:57
Messages (4)
msg242192 - (view) Author: Roy Hyunjin Han (invisibleroads) * Date: 2015-04-28 17:57
Exceptions and arguments disappear when using argparse inside a "with" statement.  The behavior was confusing and frustrating because I could not pinpoint why certain arguments were missing or unrecognized.

Unhandled exceptions inside the with statement typically trigger a traceback, unless __exit__ returns True, in which the exception is "swallowed" (

However, the NameError exception and tile_indices argument disappear because of a premature sys.exit (

$ python
Traceback (most recent call last):
  File "", line 51, in <module>
    abc  # !!!
NameError: name 'abc' is not defined
usage: [-h] --image_path PATH error: argument --image_path is required

$ python --image_path x
Traceback (most recent call last):
  File "", line 51, in <module>
    abc  # !!!
NameError: name 'abc' is not defined

$ python --image_path x --tile_indices 1
Traceback (most recent call last):
  File "", line 51, in <module>
    abc  # !!!
NameError: name 'abc' is not defined
usage: [-h] --image_path PATH error: unrecognized arguments: --tile_indices 1
from argparse import ArgumentParser

class Starter(object):

    def __init__(self):
        self.argument_parser = ArgumentParser()

    def __enter__(self):
        return self

    def __exit__(self, exception_type, exception_value, exception_traceback):

    def add_argument(self, *args, **kw):
        self.argument_parser.add_argument(*args, **kw)

with Starter() as starter:
    starter.add_argument('--image_path', metavar='PATH', required=True)
    abc  # !!!
    starter.add_argument('--tile_indices', metavar='INTEGER')
msg242196 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2015-04-28 18:29
I'm not sure why this is so surprising. parse_args in __exit__ raises a SystmExit, which the Python interpreter takes to mean the program is responsibly exiting. I don't think any other behavior would be reasonable. e.g. sys.exit() in an except clause shouldn't cause the exception to be printed.
msg242692 - (view) Author: Roy Hyunjin Han (invisibleroads) * Date: 2015-05-06 17:39
The behavior may not be surprising from a technical perspective, but it is unintuitive.

I think exceptions inside a with statement should trigger a traceback, unless you are saying that it is the responsibility of the author to catch and raise the exception inside __exit__, which feels to me like a workaround that is specific to parse_args.
msg242693 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-05-06 17:52
What is happening here is that the __exit__ method gets passed the exception, and then instead of returning and allowing the exception to propagate and be printed, it raises SystemExit (via parse_args), which causes Python to immediately shut down, *before* __exit__ returns and the exception is propagated.

So yes, you'd have to do something specific for argparse (which raises a SystemExit exception if parsing fails) if you want to do something non-normal with SystemExit.  That is, you'll need to catch SystemExit.

This is really a duplicate of issue 9938.
Date User Action Args
2015-05-06 17:52:11r.david.murraysetdependencies: + Add optional keyword argument exit_on_error to argparse.ArgumentParser
versions: + Python 3.4, Python 3.5, - Python 3.3
nosy: + r.david.murray

messages: + msg242693
resolution: not a bug -> duplicate
stage: resolved
2015-05-06 17:39:00invisibleroadssetmessages: + msg242692
2015-04-28 18:29:16benjamin.petersonsetstatus: open -> closed

nosy: + benjamin.peterson
messages: + msg242196

resolution: not a bug
2015-04-28 17:57:25invisibleroadscreate