classification
Title: Argparse exceptions should include which argument has a problem
Type: enhancement Stage:
Components: Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: paul.j3, v+python
Priority: normal Keywords:

Created on 2014-06-05 04:25 by v+python, last changed 2014-06-08 06:03 by paul.j3.

Messages (5)
msg219778 - (view) Author: Glenn Linderman (v+python) * Date: 2014-06-05 04:25
I coded up a new program, with a bunch of options, and got the following traceback when I tried to run it:

Traceback (most recent call last):
  File "D:\my\py\renmany.py", line 273, in <module>
    args = cmdl.parse_intermixed_args()
  File "D:\my\py\glu\glu.py", line 1695, in parse_intermixed_args
    args, argv = self.parse_known_intermixed_args(args, namespace)
  File "D:\my\py\glu\glu.py", line 1740, in parse_known_intermixed_args
    namespace, remaining_args = self.parse_known_args(args, namespace)
  File "C:\Python33\lib\argparse.py", line 1737, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "C:\Python33\lib\argparse.py", line 1943, in _parse_known_args
    start_index = consume_optional(start_index)
  File "C:\Python33\lib\argparse.py", line 1883, in consume_optional
    take_action(action, args, option_string)
  File "C:\Python33\lib\argparse.py", line 1811, in take_action
    action(self, namespace, argument_values, option_string)
  File "C:\Python33\lib\argparse.py", line 1015, in __call__
    parser.print_help()
  File "C:\Python33\lib\argparse.py", line 2339, in print_help
    self._print_message(self.format_help(), file)
  File "C:\Python33\lib\argparse.py", line 2323, in format_help
    return formatter.format_help()
  File "C:\Python33\lib\argparse.py", line 276, in format_help
    help = self._root_section.format_help()
  File "C:\Python33\lib\argparse.py", line 206, in format_help
    func(*args)
  File "C:\Python33\lib\argparse.py", line 206, in format_help
    func(*args)
  File "C:\Python33\lib\argparse.py", line 513, in _format_action
    help_text = self._expand_help(action)
  File "C:\Python33\lib\argparse.py", line 600, in _expand_help
    return self._get_help_string(action) % params
ValueError: unsupported format character ')' (0x29) at index 673

The only thing I can tell is that something went wrong in ArgParse. I had called a bunch of add_argument, and then a parse_known_args. I had passed parameters to the program to get a help message, so that is what I expect parse_known_args is trying to produce... and the call stack seems to confirm that. I didn't intentionally pass a format character ')' anywhere, but there are ')' characters in some of my help messages, so that is probably the source of the problem. 

No doubt I can reduce the problem space by judiciously commenting out things until I can isolate the particular help message that is causing the failure (it may be more than one as several are similar). But it seems like the exception should include the name of the argument for which the failure occurred.

[OK, I isolated, and found a "%)" sequence in one of my messages that should have been "%%)". So this is not terribly urgent, just poor reporting.]
msg219949 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-06-07 16:19
First, 'parse_intermixed_args' on stack is not relevant.  It's from an unreleased patch that we worked on.

What matters is the 'print_help', invoked probably with a '-h'.

The error message that normally specifies the problem argument is produced by ArgumentError.  The HelpFormatter does not raise such an error.  ArgumentError is usually used for parsing errors; this is a formatting one.  It's not produced by faulty commandline values.

If you must put strings like '%)` in the help line, use RawTextHelpFormatter.  Otherwise HelpFormatter assumes the help line has valid format expressions like '%(default)s'.

Or you could write your own HelpFormatter subclass with a modified '_expand_help' method, one which wraps the 'self._get_help_string(action) % params' in a 'try' block.  Probably too draconian a measure for a rare problem. :)  

It's an interesting problem, but I don't think it warrants any code changes.
msg219969 - (view) Author: Glenn Linderman (v+python) * Date: 2014-06-07 19:55
Yes, I hope someday the parse_intermixed_args patch can be released... but I know it is not relevant to this issue.

I was aware of the %(substitution_variables) in the default help formatter, but I (1) goofed and entered % without escaping it (2) was surprised at how unhelpful the Traceback was at isolating the problem.

Happily, my code had only a few instances of %) so I was able to isolate it fairly quickly, but the error report certainly shows up at quite a distance (execution-wise) from the location of the source bug.

I haven't looked at the source for the HelpFormatter code: if it concatenates all the help text and then does substitutions en masse, then it would be difficult to isolate the error to a particular argument. If, on the other hand, it loops through the help text for each argument, doing the substitutions, and later formatting and concatenating, then surrounding the substitution attempt with a try: block so that the name of the argument with the faulty help text could be reported, that would be a big help to this situation, at little cost.
msg220011 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-06-08 01:39
The ''_expand_help' method formats one action (argument) at a time, so it could issue an error message that includes that action's name.

The disconnect that you noticed arises because your bad 'help' parameter wasn't tested until is was used in a 'print_help'.

http://bugs.python.org/issue9849 asks for better testing of `nargs` (and `metavar`) values.  In the proposed patch, 'add_argument' creates a temporary HelpFormatter and tries to format a relevant portion of the help.  I suppose that test could be extended to test the 'help' parameter as well.
msg220016 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-06-08 06:03
In http://bugs.python.org/file30010/nargswarn.patch adding the '_expand_help(action)' line should test the help string (during add_argument).

    def _check_argument(self, action):
        # check action arguments
        # focus on the arguments that the parent container does not know about
        # check nargs and metavar tuple
        try:
            self._get_formatter()._format_args(action, None)
        except ValueError as e:
            raise ArgumentError(action, str(e))
        except TypeError:
            #raise ValueError("length of metavar tuple does not match nargs")
            raise ArgumentError(action, "length of metavar tuple does not match nargs")
        # check the 'help' string
        try:
            self._get_formatter()._expand_help(action)
        except (ValueError, TypeError, KeyError) as e:
            raise ArgumentError(action, 'badly formed help string')

The 'except' clause may need to be changed to capture all (or just most?) of the possible errors in the format string.  Besides your error I can imagine '%(error)s` (a KeyError).  We also need to pay attention to the differences between Py2 and Py3 errors.
History
Date User Action Args
2014-06-08 06:03:05paul.j3setmessages: + msg220016
2014-06-08 01:39:27paul.j3setmessages: + msg220011
2014-06-07 19:55:41v+pythonsetmessages: + msg219969
2014-06-07 16:19:27paul.j3setnosy: + paul.j3
messages: + msg219949
2014-06-05 04:25:32v+pythonsettype: enhancement
2014-06-05 04:25:08v+pythoncreate