This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: argparse wrapping fails with metavar="" (no metavar)
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.11, Python 3.10
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: iritkatriel, paul.j3, rhettinger, shihai1991, sjfranklin
Priority: normal Keywords: patch

Created on 2019-08-21 22:08 by sjfranklin, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 15372 open sjfranklin, 2019-08-21 23:02
Messages (6)
msg350117 - (view) Author: Sam Franklin (sjfranklin) * Date: 2019-08-21 22:08
When argparse wraps the usage text, it can fail its assertion tests with whitespace differences. This can occur when metavar="", needed if a user wishes to avoid having a metavar print. It also could occur if a   user specifies any other whitespace.
Here's a minimum example (depending on $COLUMNS):

import argparse
# based on Vajrasky Kok's script in
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--nil', metavar='', required=True)
parser.add_argument('--a', metavar='a' * 165)

This produces the AssertionError at the bottom of this comment.

A solution is to have the two asserts ignore whitespace. I'll submit a pull request very shortly for this. (First time so happy for any comments or critiques!)

A more extensive example:
import argparse
# based on Vajrasky Kok's script in
parser = argparse.ArgumentParser(prog='PROG')

parser.add_argument('--nil', metavar='', required=True)
parser.add_argument('--Line-Feed', metavar='\n', required=True)
parser.add_argument('--Tab', metavar='\t', required=True)
parser.add_argument('--Carriage-Return', metavar='\r', required=True)
                    metavar='\r\n', required=True)
parser.add_argument('--vLine-Tabulation', metavar='\v', required=True)
parser.add_argument('--x0bLine-Tabulation', metavar='\x0b', required=True)
parser.add_argument('--fForm-Feed', metavar='\f', required=True)
parser.add_argument('--x0cForm-Feed', metavar='\x0c', required=True)
parser.add_argument('--File-Separator', metavar='\x1c', required=True)
parser.add_argument('--Group-Separator', metavar='\x1d', required=True)
parser.add_argument('--Record-Separator', metavar='\x1e', required=True)
parser.add_argument('--C1-Control-Code', metavar='\x85', required=True)
parser.add_argument('--Line-Separator', metavar='\u2028', required=True)
parser.add_argument('--Paragraph-Separator', metavar='\u2029', required=True)
parser.add_argument('--a', metavar='a' * 165)

This is related to and

  File "/", line 7, in <module>
  File "/path/to/cpython/Lib/", line 1758, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/path/to/cpython/Lib/", line 1790, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/path/to/cpython/Lib/", line 1996, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/path/to/cpython/Lib/", line 1936, in consume_optional
    take_action(action, args, option_string)
  File "/path/to/cpython/Lib/", line 1864, in take_action
    action(self, namespace, argument_values, option_string)
  File "/path/to/cpython/Lib/", line 1037, in __call__
  File "/path/to/cpython/Lib/", line 2483, in print_help
    self._print_message(self.format_help(), file)
  File "/path/to/cpython/Lib/", line 2467, in format_help
    return formatter.format_help()
  File "/path/to/cpython/Lib/", line 281, in format_help
    help = self._root_section.format_help()
  File "/path/to/cpython/Lib/", line 212, in format_help
    item_help = join([func(*args) for func, args in self.items])
  File "/path/to/cpython/Lib/", line 212, in <listcomp>
    item_help = join([func(*args) for func, args in self.items])
  File "/path/to/cpython/Lib/", line 336, in _format_usage
    assert ' '.join(opt_parts) == opt_usage
msg350169 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2019-08-22 07:18
That usage formatting is extremely brittle.  It's not just "" metavar that can mess it up.  Other 'usual' characters can mess it in the same way.

The underlying problem is that it formats the whole usage, and if it is too long tries to split it into pieces, and then reassemble it in wrapped lines.  The assertion tries to verify that the split was accurate.  

Usage really needs to be rewritten in a way that keeps the individual Action pieces separate until it is ready to assemble them into final lines.  Anything else is just bandaids.
msg354647 - (view) Author: Sam Franklin (sjfranklin) * Date: 2019-10-14 19:58
Paul, very true. If I find time I may take a look at rewriting it as you suggest.
For the moment, though, there's a relatively simple change that opens up a range of allowable whitespace characters. It's still a bandage, but it covers decent-sized area and should be easily backported. What do you think of the pull request? It's passed a first review.
msg371415 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-06-12 20:28
Paul, what do you think about the PR?
msg371446 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2020-06-13 10:04
LGTM. It's a lightweight patch :)
msg404280 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-10-19 09:28
Reproduced on 3.11.
Date User Action Args
2022-04-11 14:59:19adminsetgithub: 82091
2021-10-19 09:28:30iritkatrielsetversions: + Python 3.11
nosy: + iritkatriel

messages: + msg404280

type: crash -> behavior
2020-06-13 10:04:22shihai1991setnosy: + shihai1991
messages: + msg371446
2020-06-12 20:28:45rhettingersetversions: + Python 3.10, - Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
2020-06-12 20:28:19rhettingersetnosy: + rhettinger
messages: + msg371415
2019-10-14 19:58:43sjfranklinsetmessages: + msg354647
2019-08-22 07:18:03paul.j3setmessages: + msg350169
2019-08-22 01:36:31xtreaksetnosy: + paul.j3
2019-08-21 23:02:30sjfranklinsetkeywords: + patch
stage: patch review
pull_requests: + pull_request15084
2019-08-21 22:08:19sjfranklincreate