classification
Title: document “--” as a way to distinguish option w/ narg='+' from positional argument in argparse
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Kotan, berker.peksag, bethard, docs@python, eric.smith, fdrake, gray_hemp, merwok, paul.j3
Priority: normal Keywords: easy, patch

Created on 2010-07-06 16:24 by gray_hemp, last changed 2017-01-10 00:52 by paul.j3.

Files
File name Uploaded Description Edit
issue9182.patch Kotan, 2010-11-21 09:49 review
Messages (18)
msg109402 - (view) Author: Sergey Konoplev (gray_hemp) Date: 2010-07-06 16:24
Hello,

I am starting to use argparse package and faced the problem where an optional argument w/ nargs='+' conflicts w/ a positional argument. 

Here is the test case:

>>> import argparse
>>> p = argparse.ArgumentParser()
>>> p.add_argument('foo', type=str)
_StoreAction(option_strings=[], dest='foo', nargs=None, const=None, default=None, type=<type 'str'>, choices=None, help=None, metavar=None)
>>> p.add_argument('-b', '--bar', type=int, nargs='+')
_StoreAction(option_strings=['-b', '--bar'], dest='bar', nargs='+', const=None, default=None, type=<type 'int'>, choices=None, help=None, metavar=None)
>>> p.print_help()
usage: [-h] [-b BAR [BAR ...]] foo

positional arguments:
  foo

optional arguments:
  -h, --help            show this help message and exit
  -b BAR [BAR ...], --bar BAR [BAR ...]
>>> p.parse_args('-b 123 456 bla'.split())
usage: [-h] [-b BAR [BAR ...]] foo
: error: argument -b/--bar: invalid int value: 'bla'


It prints this usage string "usage: [-h] [-b BAR [BAR ...]] foo" so it is assumed that I could use this " -b 123 456 bla" but it does not works. 

How could it be and how to solve it? 

Thank you in advance.
msg109412 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2010-07-06 17:46
Thanks for the bug report. Can you try separating option arguments from positional arguments with “--”? If yes, this would be a doc bug.

Also, can you reproduce it with Python 2.7?
msg109420 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-07-06 20:22
I get the same behavior in 2.7. Adding '--', I get:

>>> p.parse_args('-b 123 456 -- bla'.split())
Namespace(bar=[123, 456], foo='bla')

Which is what I expect.

Éric: From your comment, I'm not sure if you think it's a doc bug because '--' does work, or because it doesn't work without it. But I think it's working as designed. I'm closing this. If you think it's a doc bug, please reopen and change to the Documentation component.
msg109423 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2010-07-06 20:33
I was suggesting a doc bug only if the current doc didn’t advertise clearly this way of combining nargs with positional arguments. Using “--” is most certainly standard and right, 

Using “--” to explicitly signal positional arguments is most certainly right and standard, but people have to read about that for the first time somewhere. In a perfect world, that would be in their shell’s man page, but better add a word about it in the reference. The “--” marker is actually mentioned in one section: http://docs.python.org/dev/library/argparse#arguments-containing

I’m reopening, suggesting that a line be added to explain the standard “--” separator for use cases like Sergey’s.
msg109425 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2010-07-06 20:34
Duh, please ignore the first two lines, I forgot to remove them after rephrasing my comment. Sorry.
msg109426 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2010-07-06 20:40
On Tue, Jul 6, 2010 at 4:33 PM, Éric Araujo <report@bugs.python.org> wrote:
> Using “--” to explicitly signal positional arguments is most certainly
> right and standard, but people have to read about that for the first
> time somewhere. In a perfect world, that would be in their shell’s man
> page, but better add a word about it in the reference.

It's a convention; not all applications in the larger world support it at all.

It needs to be documented for argparse, and also for specific applications
(which may or may not be built using argparse or Python.

This does not belong in the shell documentation unless needed for the shell
itself.
msg109427 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2010-07-06 20:43
I always forget that not all programs follow getopt-style parsing rules, thanks for the reminder.
msg109431 - (view) Author: Sergey Konoplev (gray_hemp) Date: 2010-07-06 21:51
Thank you for the hint. It is realy not so obvious.

May be it is worth to add "--" into the usage string this way

usage: [-h] [-b BAR [BAR ...]] -- foo

Otherwise it leads to misunderstanding.
msg121519 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2010-11-19 14:32
Sergey: Do you want to make a patch for that, and/or for the documentation?  Guidelines are on http://www.python.org/dev/patches/
msg121532 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-11-19 15:20
The original point is basically a duplicate of issue 9338. It is undesirable behavior, I just don't know how to fix it. Patches to fix it are welcome (on issue 9338). ;-)

As to documenting '--', I agree it's hidden too far down in the documentation currently. I'd be happy to approve any documentation patch that makes the '--' workaround more visible.
msg121534 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2010-11-19 15:23
You’re right, sorry I was unclear: “a patch for that” referred to the addition of “--” in the generated usage text.
msg121724 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 19:08
I did the patch for adding the "--" in the generated usage text inside issue 9338.
msg121850 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-11-21 03:17
Sorry, I think I confused you, please post that patch here. This issue is for documenting the workarounds, issue 9338 is for actually solving the problem.

I glanced at your patch on that issue, and it looks basically okay, though I'd like to see a few more tests for the help messages to cover things like:

   -x X A                       # no --
   -x [X [X ...]] -y Y [Y ...]  # no --
   -x [X] -- A B                # needs --
   -x X [X ...]                 # needs --
   -x [X [X ...]]               # needs --

It's possible the first two are already covered by the other tests, but at least the last three should all be tested for.
msg121891 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-21 09:44
Steven: From msg121850 I think the last two examples do not need the "--". Since there are no positional arguments, the "--" is not needed. However the following additional cases exist:

-x X [X ...] -y Y -- A B # since optionals might come in other order

Basically every of the three variable length classes need to be tested: ZeroOrOne, ZeroOrMore and OneOrMore. This might always come in a conflict with positional arguments needed afterward (even with optional positional arguments!).

BTW - I recognized the following seem to be supported:

for [-x [X ...]] -- [y ...] 
the following input: a -x -- b

I would expect to be 'a' and 'b' positional arguments and the optional '-x' has no argument. However this seem not to work. Another issue or desired behavior?
msg121893 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-21 09:46
Added more unit tests for testing the help text as well as the functionality of the "--" separator.
msg121935 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-11-21 15:11
Yeah, sorry, those last two should have had arguments after them.

I think we have to stick with the current behavior - otherwise there's no way to do something like -x -A -B, where you want -A and -B to be arguments to -x. (You can get that now with -x -- -A -B.)
msg121950 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-21 16:05
Steven: The last part I guess would belong better in issue 9334. Using a "--" to end all optionals is just a convention but "--" could also be the argument to some option as well. We need to limit this in any way, and the way it is done right now seem very reasonable for me.

Adding some additional unit tests testing this limits might be beneficial but I guess would be best done during issue 9334.
msg166243 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-07-23 18:44
Ok, here's what I think needs to go into the documentation here:

(1) Add a separate section to the argparse docs about '--'. Give examples like the ones in this issue, and show how '--' can solve them

(2) Cross-reference the section on '--' from nargs='?', nargs='*', nargs='+' and the "Arguments containing -" sections (and any other places where people are likely to run into the need for '--').

All of these changes should be on Doc/library/argparse.rst - I don't think we want to change the code at the moment.
History
Date User Action Args
2017-01-10 00:52:33paul.j3setnosy: + paul.j3
2017-01-08 06:09:09berker.peksagsetnosy: + berker.peksag
2016-01-27 02:21:19martin.panterlinkissue26181 superseder
2012-07-23 18:44:03bethardsetmessages: + msg166243
2010-11-21 16:05:24Kotansetmessages: + msg121950
2010-11-21 15:11:20bethardsetmessages: + msg121935
2010-11-21 09:49:05Kotansetfiles: + issue9182.patch
2010-11-21 09:48:59Kotansetfiles: - issue9182.patch
2010-11-21 09:46:58Kotansetfiles: + issue9182.patch
keywords: + patch
messages: + msg121893
2010-11-21 09:44:06Kotansetmessages: + msg121891
2010-11-21 03:17:50bethardsetmessages: + msg121850
2010-11-20 19:08:57Kotansetnosy: + Kotan
messages: + msg121724
2010-11-19 15:23:12merwoksetmessages: + msg121534
2010-11-19 15:20:41bethardsetmessages: + msg121532
versions: - Python 3.1
2010-11-19 14:32:16merwoksetversions: + Python 3.1, Python 3.2, - Python 2.6
nosy: + bethard

messages: + msg121519

resolution: accepted ->
2010-07-06 21:51:21gray_hempsetmessages: + msg109431
2010-07-06 20:43:17merwoksetkeywords: + easy
resolution: accepted
messages: + msg109427

title: argparse: optional argument w/ narg='+' conflicts w/ positional argsument -> document “--” as a way to distinguish option w/ narg='+' from positional argument in argparse
2010-07-06 20:40:52fdrakesetnosy: + fdrake
messages: + msg109426
2010-07-06 20:34:49merwoksetmessages: + msg109425
2010-07-06 20:33:52merwoksetstatus: closed -> open

assignee: docs@python
components: + Documentation, - Library (Lib)

nosy: + docs@python
messages: + msg109423
resolution: not a bug -> (no value)
stage: needs patch
2010-07-06 20:22:07eric.smithsetstatus: open -> closed
versions: + Python 2.7
nosy: + eric.smith

messages: + msg109420

resolution: not a bug
2010-07-06 17:46:09merwoksetnosy: + merwok
messages: + msg109412
2010-07-06 16:24:38gray_hempcreate