classification
Title: argparse wrapping fails with metavar="" (no metavar)
Type: crash Stage: patch review
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: paul.j3, sjfranklin
Priority: normal Keywords: patch

Created on 2019-08-21 22:08 by sjfranklin, last changed 2019-10-14 19:58 by sjfranklin.

Pull Requests
URL Status Linked Edit
PR 15372 open sjfranklin, 2019-08-21 23:02
Messages (3)
msg350117 - (view) Author: Sam Franklin (sjfranklin) * Date: 2019-08-21 22:08
When argparse wraps the usage text, it can fail its assertion tests with whitespace differences. This can occur when metavar="", needed if a user wishes to avoid having a metavar print. It also could occur if a   user specifies any other whitespace.
Here's a minimum example (depending on $COLUMNS):

import argparse
# based on Vajrasky Kok's script in https://bugs.python.org/issue11874
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--nil', metavar='', required=True)
parser.add_argument('--a', metavar='a' * 165)
parser.parse_args()

This produces the AssertionError at the bottom of this comment.

A solution is to have the two asserts ignore whitespace. I'll submit a pull request very shortly for this. (First time so happy for any comments or critiques!)

A more extensive example:
import argparse
# based on Vajrasky Kok's script in https://bugs.python.org/issue11874
parser = argparse.ArgumentParser(prog='PROG')

parser.add_argument('--nil', metavar='', required=True)
parser.add_argument('--Line-Feed', metavar='\n', required=True)
parser.add_argument('--Tab', metavar='\t', required=True)
parser.add_argument('--Carriage-Return', metavar='\r', required=True)
parser.add_argument('--Carriage-Return-and-Line-Feed',
                    metavar='\r\n', required=True)
parser.add_argument('--vLine-Tabulation', metavar='\v', required=True)
parser.add_argument('--x0bLine-Tabulation', metavar='\x0b', required=True)
parser.add_argument('--fForm-Feed', metavar='\f', required=True)
parser.add_argument('--x0cForm-Feed', metavar='\x0c', required=True)
parser.add_argument('--File-Separator', metavar='\x1c', required=True)
parser.add_argument('--Group-Separator', metavar='\x1d', required=True)
parser.add_argument('--Record-Separator', metavar='\x1e', required=True)
parser.add_argument('--C1-Control-Code', metavar='\x85', required=True)
parser.add_argument('--Line-Separator', metavar='\u2028', required=True)
parser.add_argument('--Paragraph-Separator', metavar='\u2029', required=True)
parser.add_argument('--a', metavar='a' * 165)
parser.parse_args()

This is related to https://bugs.python.org/issue17890 and https://bugs.python.org/issue32867.

  File "/minimum_argparse_bug.py", line 7, in <module>
    parser.parse_args()
  File "/path/to/cpython/Lib/argparse.py", line 1758, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/path/to/cpython/Lib/argparse.py", line 1790, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/path/to/cpython/Lib/argparse.py", line 1996, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/path/to/cpython/Lib/argparse.py", line 1936, in consume_optional
    take_action(action, args, option_string)
  File "/path/to/cpython/Lib/argparse.py", line 1864, in take_action
    action(self, namespace, argument_values, option_string)
  File "/path/to/cpython/Lib/argparse.py", line 1037, in __call__
    parser.print_help()
  File "/path/to/cpython/Lib/argparse.py", line 2483, in print_help
    self._print_message(self.format_help(), file)
  File "/path/to/cpython/Lib/argparse.py", line 2467, in format_help
    return formatter.format_help()
  File "/path/to/cpython/Lib/argparse.py", line 281, in format_help
    help = self._root_section.format_help()
  File "/path/to/cpython/Lib/argparse.py", line 212, in format_help
    item_help = join([func(*args) for func, args in self.items])
  File "/path/to/cpython/Lib/argparse.py", line 212, in <listcomp>
    item_help = join([func(*args) for func, args in self.items])
  File "/path/to/cpython/Lib/argparse.py", line 336, in _format_usage
    assert ' '.join(opt_parts) == opt_usage
AssertionError
msg350169 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2019-08-22 07:18
That usage formatting is extremely brittle.  It's not just "" metavar that can mess it up.  Other 'usual' characters can mess it in the same way.

The underlying problem is that it formats the whole usage, and if it is too long tries to split it into pieces, and then reassemble it in wrapped lines.  The assertion tries to verify that the split was accurate.  

Usage really needs to be rewritten in a way that keeps the individual Action pieces separate until it is ready to assemble them into final lines.  Anything else is just bandaids.
msg354647 - (view) Author: Sam Franklin (sjfranklin) * Date: 2019-10-14 19:58
Paul, very true. If I find time I may take a look at rewriting it as you suggest.
For the moment, though, there's a relatively simple change that opens up a range of allowable whitespace characters. It's still a bandage, but it covers decent-sized area and should be easily backported. What do you think of the pull request? It's passed a first review.
History
Date User Action Args
2019-10-14 19:58:43sjfranklinsetmessages: + msg354647
2019-08-22 07:18:03paul.j3setmessages: + msg350169
2019-08-22 01:36:31xtreaksetnosy: + paul.j3
2019-08-21 23:02:30sjfranklinsetkeywords: + patch
stage: patch review
pull_requests: + pull_request15084
2019-08-21 22:08:19sjfranklincreate