This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: argparse allows nargs>1 for positional arguments but doesn't allow metavar to be a tuple
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bethard, cykerway, iritkatriel, paul.j3, rhettinger, shihai1991, tshepang, vvas
Priority: normal Keywords: patch

Created on 2012-02-21 17:38 by tshepang, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
issue14074.patch paul.j3, 2013-07-26 20:20 review
issue14074_1.patch paul.j3, 2014-04-27 06:24 review
14074.patch cykerway, 2018-12-01 13:50 lightweight patch
Pull Requests
URL Status Linked Edit
PR 10847 open python-dev, 2018-12-01 13:47
Messages (13)
msg153884 - (view) Author: Tshepang Lekhonkhobe (tshepang) * Date: 2012-02-21 17:38
Here's four commands to demonstrate my problem:

My sample code:
$ cat prog.py 
import argparse
parse = argparse.ArgumentParser()
parse.add_argument("foo", nargs=2, metavar=("foo", "bar"))
args = parse.parse_args()
print(args)

This output could be friendlier:
$ python3 prog.py --help
Traceback (most recent call last):
  File "prog.py", line 4, in <module>
    args = parse.parse_args()
  File "/home/wena/cpython/Lib/argparse.py", line 1722, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/home/wena/cpython/Lib/argparse.py", line 1754, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/home/wena/cpython/Lib/argparse.py", line 1960, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/home/wena/cpython/Lib/argparse.py", line 1900, in consume_optional
    take_action(action, args, option_string)
  File "/home/wena/cpython/Lib/argparse.py", line 1828, in take_action
    action(self, namespace, argument_values, option_string)
  File "/home/wena/cpython/Lib/argparse.py", line 1015, in __call__
    parser.print_help()
  File "/home/wena/cpython/Lib/argparse.py", line 2347, in print_help
    self._print_message(self.format_help(), file)
  File "/home/wena/cpython/Lib/argparse.py", line 2314, in format_help
    formatter.add_arguments(action_group._group_actions)
  File "/home/wena/cpython/Lib/argparse.py", line 270, in add_arguments
    self.add_argument(action)
  File "/home/wena/cpython/Lib/argparse.py", line 255, in add_argument
    invocations = [get_invocation(action)]
  File "/home/wena/cpython/Lib/argparse.py", line 533, in _format_action_invocation
    metavar, = self._metavar_formatter(action, default)(1)
ValueError: too many values to unpack (expected 1)

On Python 3.2 (from Debian), I get "error: too few arguments". But with latest mercurial I get:
$ python3 prog.py
Traceback (most recent call last):
  File "prog.py", line 4, in <module>
    args = parse.parse_args()
  File "/home/wena/cpython/Lib/argparse.py", line 1722, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/home/wena/cpython/Lib/argparse.py", line 1754, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/home/wena/cpython/Lib/argparse.py", line 1973, in _parse_known_args
    ', '.join(required_actions))
TypeError: sequence item 0: expected str instance, tuple found

But this looks okay:
$ python3 prog.py a b
Namespace(foo=['a', 'b'])
msg154236 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-02-25 10:45
Looks like the problem is that "_format_action_invocation" is not being as careful with the different possibilities for metavar as "_format_args" is.  Patches welcome!
msg193737 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-07-26 20:20
This patch fixes the problem by joining the metavar terms with '|'.  So the help for the test case (adapted from an existing tuple test) looks like:

    usage: PROG [-h] W1 [W2 ...] [X1 [X2 ...]] Y1 Y2 Y3 [Z1]
    positional arguments:
      W1|W2       w
      X1|X2       x
      Y1|Y2|Y3    y
      Z1          z

Alternatives include:
- use ',',' ',or '/' instead of '|'
- use just the 1st item of the tuple
- use the default (non-tuple) metavar in the help line
These all pass existing tests.

The last alternative would use:

    #metavar = '|'.join(metavar)
    if len(metavar)>1:
        metavar = default
    else:
        metavar = metavar[0]
msg217184 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-04-26 06:35
oops - to fix the error message that OP complained about, I need to patch '_get_action_name' as well:

    def _get_action_name(argument):
    ...
        elif argument.metavar not in (None, SUPPRESS):
            metavar = argument.metavar
            if isinstance(metavar, tuple):
                metavar = '|'.join(metavar)
            return metavar
msg217240 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-04-27 06:24
This patch fixes both help and error formatting.

A module level '_format_metavars' does the formatting for both.

I have tried several alternatives, including using the 'usage' style.

There is similarity between this fix and that for issue 16468 (custom choices), though I don't think they conflict.  In both cases, code needed to format the usage or help is also needed to help format error messages.

Issue 9849 (better nargs warning) is another case where error checking in the parser depends on the formatter.  In the long run we may want to refactor these issues.
msg286227 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2017-01-25 05:38
An alternative fix is to disallow tuple metavars for positionals.  

A tuple metavar may look nice in the usage.  But a well selected dest seems better in the help line, and error messages.  Using dest in all 3 roles, as we do now, is the simplest compromise.
msg330854 - (view) Author: Cyker Way (cykerway) * Date: 2018-12-01 13:50
Can confirm this bug still exists on master branch, python3.7, python3.6, and very likely other versions since it's reported.

It seems only `_format_action_invocation` and `_get_action_name` need to be fixed. So we can do it more lightweight (<10 lines).
msg360841 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-01-28 07:52
-0 on going forward with this.  AFAICT, no one has requested tuple support (the OP just wanted better error handling).  And more fined grained control can already be had just by specifying separate positional arguments.

Paul suggested that it may be a better plan to just disallow tuple metavars and to give a better error message as the OP suggested.  That makes sense to me.
msg361154 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2020-02-01 07:43
I am not sure disallowing tuple metavars would break backup compatibility or not~
msg361184 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-02-01 19:03
> I am not sure disallowing tuple metavars would break backup 
> compatibility or not~

Given that tuple metavars currently doesn't work, there is no backward compatibility issue.  What is being proposed is to give more friendly error messages.
msg361215 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2020-02-02 05:44
> Given that tuple metavars currently doesn't work, there is no backward compatibility issue.  What is being proposed is to give more friendly error messages.

Got it, thanks.

> Paul suggested that it may be a better plan to just disallow tuple metavars and to give a better error message as the OP suggested.  That makes sense to me.

+1
msg361336 - (view) Author: Cyker Way (cykerway) * Date: 2020-02-04 02:53
Tuple support is documented:

https://github.com/python/cpython/commit/98047eb806227f11212f6a42c242030a51964e30#diff-9c4a053d29149ba40370fbdddd3e34faR1059

https://docs.python.org/3/library/argparse.html#metavar

>   Providing a tuple to ``metavar`` specifies a different display for each of the arguments:

From his word here I feel like our professor meant to support tuple metavar but the implementation was not careful. He called for a patch eight years ago. Let's summon him and see if he has changed his mind?
msg407109 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-26 23:42
Reproduced on 3.11, with a different error:

% ./python.exe prog.py
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython-1/prog.py", line 5, in <module>
    args = parse.parse_args()
           ^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/argparse.py", line 1822, in parse_args
    args, argv = self.parse_known_args(args, namespace)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/argparse.py", line 1855, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/iritkatriel/src/cpython-1/Lib/argparse.py", line 2093, in _parse_known_args
    ', '.join(required_actions))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, tuple found
History
Date User Action Args
2022-04-11 14:57:26adminsetgithub: 58282
2021-11-26 23:42:24iritkatriellinkissue21616 superseder
2021-11-26 23:42:00iritkatrielsetversions: + Python 3.10, Python 3.11
nosy: + iritkatriel

messages: + msg407109

type: behavior
2020-02-04 02:53:59cykerwaysetmessages: + msg361336
2020-02-02 05:44:51shihai1991setversions: + Python 3.9, - Python 3.5
2020-02-02 05:44:04shihai1991setmessages: + msg361215
2020-02-01 19:03:23rhettingersetmessages: + msg361184
2020-02-01 07:43:27shihai1991setnosy: + shihai1991

messages: + msg361154
versions: + Python 3.5, - Python 3.3
2020-01-28 07:52:46rhettingersetpriority: high -> normal
nosy: + rhettinger
messages: + msg360841

2018-12-01 13:50:50cykerwaysetfiles: + 14074.patch
nosy: + cykerway
messages: + msg330854

2018-12-01 13:47:05python-devsetstage: patch review
pull_requests: + pull_request10082
2017-01-25 05:38:09paul.j3setmessages: + msg286227
2017-01-24 23:33:07paul.j3setpriority: normal -> high
2014-06-12 18:40:24vvassetnosy: + vvas
2014-04-27 06:24:21paul.j3setfiles: + issue14074_1.patch

messages: + msg217240
2014-04-26 06:35:03paul.j3setmessages: + msg217184
2013-07-26 20:20:50paul.j3setfiles: + issue14074.patch
keywords: + patch
messages: + msg193737
2013-07-26 17:21:46paul.j3setnosy: + paul.j3
2012-02-25 10:45:07bethardsetmessages: + msg154236
2012-02-25 07:41:29tshepangsettitle: argparse does not allow nargs>1 for positional arguments but doesn't allow metavar to be a tuple -> argparse allows nargs>1 for positional arguments but doesn't allow metavar to be a tuple
2012-02-25 00:35:35terry.reedysetnosy: + bethard
2012-02-21 17:38:38tshepangcreate