This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: argparse: subparsers, argument abbreviations and ambiguous option
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bethard, eric.araujo, jakub, paul.j3, tshepang
Priority: normal Keywords: patch

Created on 2012-03-18 20:27 by jakub, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
argparse_subparsers_ambiguous_bug.py jakub, 2012-03-18 20:27 Bug reproduction
argparse_dirty_hack.py jakub, 2012-03-19 18:10
subparser_optionals.diff bethard, 2012-07-21 20:02 review
subparser_patch.diff paul.j3, 2013-09-29 04:30 review
Messages (14)
msg156272 - (view) Author: Jakub Warmuz (jakub) Date: 2012-03-18 20:27
Assuming following:
1. optional argument, say "--foo", in a subparser;
2. at least two different optional arguments in the main parser prefixed with "--foo", say "--foo1" and "--foo2";

parsing fails with "error: ambiguous option: --foo could match --foo1, --foo2".

It seems like argument abbreviation mechanism described at http://docs.python.org/library/argparse.html#argument-abbreviations is not working properly when in use with subparsers...
msg156279 - (view) Author: Tshepang Lekhonkhobe (tshepang) * Date: 2012-03-18 22:21
More or less a duplicate of 12713?
msg156280 - (view) Author: Tshepang Lekhonkhobe (tshepang) * Date: 2012-03-18 22:21
Sorry, I meant #12713.
msg156290 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-03-18 23:16
Yep. Closing as duplicate.
msg156302 - (view) Author: Jakub Warmuz (jakub) Date: 2012-03-19 01:27
I don't understand how both bugs are related. Surely, patch provided for #12713 does not fix the issue I described.
msg156304 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-03-19 01:39
My mistake. I see that the error you're getting is a bad interaction between the option in the main parser and an ambiguous option in the subparser.
msg156306 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-03-19 01:51
The problem is basically that _parse_known_args calls _parse_optional to determine whether something is an optional or a positional. But _parse_optional only checks "if arg_string in self._option_string_actions", that is, if the string can be found in the main parser actions.

So either this method needs to check the subparser actions, or the "if arg_string in self._option_string_actions" stuff needs to be delayed until the subparser. Neither of these seems like a simple change, but I agree it's a bug.
msg156352 - (view) Author: Jakub Warmuz (jakub) Date: 2012-03-19 18:10
Attached quick&dirty fix I'm currently using in my project. Maybe this will help to create a valid patch.

Instead of changing _parse_optional, I've overridden _get_option_tuples to scan subparsers for matching optional arguments if option is ambiguous in the current parser.

I've only skimmed through the code, so forgive me if that fix is useless.
msg166059 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-07-21 20:02
I think the real fix needs to search recursively though all subparsers. Here's a first draft of a patch that does this. I believe your sample code works after this patch.

I don't have the time to turn your bug report into proper test (and add a test for the recursive bit), but hopefully this is enough to let someone else finish up fixing this bug.

And yes, your "dirty hack" was still helpful in putting this together. ;-)
msg198456 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-09-26 23:04
Steven's patch (subparse_optionals.diff) run with jakub's test case (argparse_subparses_ambiguous_bug.py) works.  But if the input string is 

    print(parser.parse_args('--foo baz'.split()))

produces

    Namespace(cmd=None, foo='baz', foo1=None, foo2=None)

(I added the 'cmd' subparse 'dest').  

Two things seem to be going on now: 

1) '--foo' is being parsed even though its subparser is not invoked, 

2) and the subparser is not required.

The issue of whether subparsers are required or not is another issue.  They used to be required, but the testing for 'required' was changed, and subparsers fell through the crack.

I suspect that if the missing subparser error were raised, the first issue wouldn't be apparent.
msg198507 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-09-28 06:32
I think the correction to the problem that I noted in the previous post is to return 'None, arg_string, None', rather than 'action, arg_string, None' in the case where the action is found in a subparser.

This is what '_parse_optional' does at the end:

        # it was meant to be an optional but there is no such option
        # in this parser (though it might be a valid option in a subparser)
        return None, arg_string, None

An input like '--foo baz' would then produce an 'invalid choice' error.  Since '--foo' is an optional that the primary parser does not recognize, 'baz' in interpreted as a positional, in this case an invalid subparser choice.

I'm working on cleaning up a test script.
msg198508 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-09-28 06:35
In the last patch, 'parser.scan = True' is needed to activate this fix.  I left that switch in for testing convenience.
msg198563 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-09-29 04:30
This the argparse patch as described in the previous post, along with test_argparse tests.  

For now the tests handle both subparser required cases and not required ones ( http://bugs.python.org/issue9253 ).  While error messages can differ based on this requirement, it does not affect the 'ambiguity' that underlies the current issue.
msg225044 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-08-07 22:47
Another issue dealing with abbreviations is close to being committed.

http://bugs.python.org/issue14910
argparse: disable abbreviation
History
Date User Action Args
2022-04-11 14:57:28adminsetgithub: 58573
2014-08-07 22:47:31paul.j3setmessages: + msg225044
2013-09-29 04:30:28paul.j3setfiles: + subparser_patch.diff

messages: + msg198563
2013-09-29 04:24:10paul.j3setfiles: - subparser_patch.diff
2013-09-28 06:35:15paul.j3setmessages: + msg198508
2013-09-28 06:32:35paul.j3setfiles: + subparser_patch.diff

messages: + msg198507
2013-09-26 23:04:19paul.j3setmessages: + msg198456
2013-09-16 06:59:41paul.j3setnosy: + paul.j3
2012-07-21 20:02:49bethardsetfiles: + subparser_optionals.diff
keywords: + patch
messages: + msg166059
2012-03-24 15:24:41eric.araujosetnosy: + eric.araujo
stage: test needed

versions: + Python 3.3
2012-03-19 18:10:16jakubsetfiles: + argparse_dirty_hack.py

messages: + msg156352
2012-03-19 01:51:54bethardsetmessages: + msg156306
2012-03-19 01:39:04bethardsetstatus: closed -> open
resolution: duplicate ->
superseder: argparse: allow abbreviation of sub commands by users ->
messages: + msg156304
2012-03-19 01:27:26jakubsetmessages: + msg156302
2012-03-18 23:16:57bethardsetstatus: open -> closed
resolution: duplicate
superseder: argparse: allow abbreviation of sub commands by users
messages: + msg156290
2012-03-18 22:21:27tshepangsetmessages: + msg156280
2012-03-18 22:21:07tshepangsetmessages: + msg156279
2012-03-18 22:16:25tshepangsetnosy: + tshepang
2012-03-18 20:27:59jakubcreate