classification
Title: argparse: Allow the use of -- to break out of nargs and into subparser
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bethard, elsdoerfer, eric.araujo, paul.j3, r.david.murray
Priority: normal Keywords: patch

Created on 2010-08-11 19:21 by elsdoerfer, last changed 2014-07-07 17:12 by paul.j3.

Files
File name Uploaded Description Edit
issue9571_1.patch paul.j3, 2014-07-07 17:12 review
Messages (5)
msg113616 - (view) Author: Michael.Elsdörfer (elsdoerfer) Date: 2010-08-11 19:21
argparse already seems to support -- to indicate that what follows are positional arguments. However, I would like to parse something like:

./script.py --ignore one two -- COMMAND

I.e., --ignore is an nargs='+' argument, and I need a way to break out of --ignore and have argparse consider what follows on it's own merits. If COMMAND in the above example refers to a subparser, this won't work:

error: invalid choice: '--' (choose from 'command1', 'command2', 'command3')

I'm not sure what's the best solution here. Allowing -- here would change the semantics of forcing everything that follows to be positional arguments, since the subparser might have flags. I'm not sure if that is what is required by Unix conventions, but if not, then I think it makes sense to allow -- to be followed by a subparser.
msg113647 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-08-12 01:45
It looks like, if accepted, this would be a feature request,so I'm marking it as such and setting versions to 3.2.  You'd have to provide a patch pretty soon to get it in to 3.2, though.

However, I'm guessing that this is something better off implemented via subclassing in your code, since it sounds like a fairly unusual command pattern.  (I've only ever seen options after a -- in a unix comand when those options were being passed unparsed to some *other* command that the first command was a wrapper for.)

I've added Steven as nosy, we'll see what he thinks.
msg113668 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-08-12 08:43
This is closely related to issue 9338. The parser should know that your command line requires at least the COMMAND argument, so it should stop parsing in time for that. However, in the case of subcommands, even if we solved issue 9338, you would still get the behavior that

./script.py --ignore one two COMMAND arg1 arg2

would get parsed as "arg2" being the command. So I guess there still ought to be a way to tell argparse to stop parsing nargs='+' optionals.

Seems like there's also a bug in the current behavior - you should get an error saying that no command was given, not an error saying you issued the command "--".
msg222427 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-07-07 00:18
In elsdoerfer's example, the '--' effectively ends the argument list for '--ignore'.  '--' are not allowed in the arguments of an optional (see the end of '_get_nargs_pattern()').

Rather the problem is at the start of _get_values()

        if action.nargs not in [PARSER, REMAINDER]:
            arg_strings = [s for s in arg_strings if s != '--']

'--' is not stripped out of the 'parser' input, hence the error message:

    error: invalid choice: '--'

You can see this by replacing the subparsers with an argument with 'nargs=PARSER'.  The argument will get `['--','COMMAND',...]`.

http://bugs.python.org/issue13922 tries to rework how '--' are handled.  Ideally only the current '--' should be removed, leaving the rest to be handled by the subparser (or whoever else gets the strings).

Looks like the 13922 fix needs another fix, one that removes '--' if it is the 1st string for a REMAINDER or PARSER argument.

With that fix:

    ./script.py --ignore one two -- COMMAND arg1 arg2

should work, assigning ['one','two'] to 'ignore', and ['arg1','arg2'] to 'COMMAND's positional.

(I've tested this in a development version with many other changes.  I'll try to write a simpler patch.)

----------------

If a 'end of a list' flag is still needed (as between 2 * positionals), a 'counter' or 'store_true' optional could be used.  Or a new action class that doesn't write anything to the namespace could be written.
msg222478 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-07-07 17:12
This patch modifies that '--' handling from 13922, removing '--' from PARSER and REMAINDER if it is the first string:

        if action.nargs not in [PARSER, REMAINDER]:
            try:
                arg_strings.remove('--')
            except ValueError:
                pass
        else:
            # remove '--' if it is the first string, issue9571
            if arg_strings and arg_strings[0] == '--':
                arg_strings = arg_strings[1:]

Doing this for PARSER addresses this issue, and should not have any backward compatibility issues, since starting a 'A...' list with '--' does not make sense.

The argument for doing this with REMAINDER is a bit weaker.  I can't think of why a user would expect or want an initial '--', but someone might have written code to compensate for this flaw.

test_argparse has 2 new tests, added at the end of the subparsers class.  The one for subparser has to contend with a bug that makes subparsers optional (http://bugs.python.org/issue9253).  The REMAINDER one is clearly related, though it doesn't fit the test class theme.
History
Date User Action Args
2014-07-07 17:12:55paul.j3setfiles: + issue9571_1.patch
keywords: + patch
messages: + msg222478
2014-07-07 00:18:27paul.j3setmessages: + msg222427
2014-07-05 23:41:26BreamoreBoysetnosy: + paul.j3

versions: + Python 3.5, - Python 3.2
2010-09-20 21:33:29eric.araujosetnosy: + eric.araujo
2010-08-12 08:43:07bethardsetmessages: + msg113668
2010-08-12 01:45:02r.david.murraysetversions: - Python 3.1, Python 2.7, Python 3.3
nosy: + r.david.murray, bethard

messages: + msg113647

type: behavior -> enhancement
2010-08-11 19:21:25elsdoerfercreate