classification
Title: argparse: allow abbreviation of sub commands by users
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bethard, chriz, eric.araujo, ned.deily, paul.j3, pwil3058, python-dev, tshepang, vinay.sajip, zach.ware
Priority: normal Keywords: needs review

Created on 2011-08-09 01:55 by pwil3058, last changed 2016-08-23 07:54 by vinay.sajip.

Files
File name Uploaded Description Edit
abbrev_subcmds.patch chriz, 2011-08-20 08:49 Patch to make subcommands abbreviable (2.7.1)
abbrev_subcmds_3.3.patch chriz, 2011-08-21 16:29 Patch to make subcommands abbreviable (3.3) review
abbrev_subcmds_plus_tests_3.3.patch chriz, 2011-08-22 17:04 Patch to make subcommands abbreviable (3.3) - now with tests review
abbrev_subcmds_incl_tests_3.4.patch chriz, 2013-02-27 20:26 Patch to make subcommands abbreviable (3.4) review
abbr_bug_test.diff zach.ware, 2016-08-19 18:36 review
Messages (26)
msg141805 - (view) Author: Peter Williams (pwil3058) Date: 2011-08-09 01:55
Quite often the names of subcommands are quite long so that their meaning is clear.  However, the downside to this is the increased typing the user of the command must make.  A compromise between clarity and brevity can be realized if abbreviation of subcommands is allowed with the proviso that the abbreviation must be long enough to remove any ambiguity.

So that this feature is not foisted on every programmer that uses argparse it could be made an option to the ArgumentParser class with the (possible) additional refinement of specifying a minimum number of characters for the abbreviations.
msg141968 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-12 16:20
Interesting idea.  I know that with Mercurial for example, I use abbreviations and aliases all the time.  Note that argparse already has aliases (or there is an open feature request about it).

Steven: What do you think of this request?
msg142492 - (view) Author: Christian Ziemski (chriz) Date: 2011-08-19 19:30
I just made such a change to Python 2.7's argparse.
If there is interest I'll post a patch here.

Unfortunately I can't find the description how to produce a proper patch.
The link I found (http://www.python.org/dev/patches/) gives an error 404.
msg142513 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2011-08-20 05:40
http://docs.python.org/devguide/patch.html
msg142518 - (view) Author: Christian Ziemski (chriz) Date: 2011-08-20 08:49
I attached a patch against argparse.py from Python 2.7.1.

Subcommmands can now be abbreviated as long as they are unambiguous.
Otherwise an error message 'ambigous choice' will be thrown (like the 'invalid choice' one).

(It's my first patch, so hopefully I did it right.)

BTW: Éric mentioned aliases for argparse. 
For that I have done a patch too. 
I'll try to find the appropriate issue number here to post it.
msg142519 - (view) Author: Christian Ziemski (chriz) Date: 2011-08-20 09:02
Since there seems to be no means to edit (my last) message a little followup regarding aliases:

I found http://bugs.python.org/issue9234 
"argparse: aliases for positional arguments (subparsers)"

That one is for version 3.2 and already closed. 
So I'll stop bugging here with aliases.
msg142567 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-20 21:20
Yes, I checked and aliases are already supported.

You seem to have overlooked the Versions field on the top of this page: new features don’t go in stable releases, so your patch would have to apply to default, a.k.a. 3.3.
msg142592 - (view) Author: Christian Ziemski (chriz) Date: 2011-08-21 06:08
Just for my understanding:
Even "the current development/testing version: Python 3.2.2 rc 1 (August 14, 2011)" isn't new enough here?

So I'll have to find out how to install Python 3.3 in parallel to my 2.7.1 (on Fedora 15).

Starting with "hg clone http://hg.python.org/cpython" now.

Until later (after reading http://docs.python.org/devguide/ by myself of course ;-)
msg142598 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-21 08:18
> Even "the current development/testing version: Python 3.2.2 rc 1
> (August 14, 2011)" isn't new enough here?
No.  (Where does the quote come from?)  The current development version is 3.3; its version number is currently 3.3.0a0, to mean it was not released yet.  3.2 is a stable release now; 3.2.2c1 is the candidate release of the next bugfix version of 3.2, which is one of the stable versions (with 2.7).

> So I'll have to find out how to install Python 3.3 in parallel to my 2.7.1
> Starting with "hg clone http://hg.python.org/cpython" now.
Yep, there’s no need to install.  Cloning http://hg.python.org/cpython#default will get you 3.3.  If you already have a clone for 3.2, you can also pull the default branch in your existing clone; feel free to ask for anything that’s not in the devguide or not clear.
msg142599 - (view) Author: Christian Ziemski (chriz) Date: 2011-08-21 09:03
>> Even "the current development/testing version: Python 3.2.2 rc 1
>> (August 14, 2011)" isn't new enough here?
>
> No.  (Where does the quote come from?) 

http://python.org/download/releases/

> Cloning http://hg.python.org/cpython#default will get you 3.3

I have 3.3 compiled and running now. :-)

> feel free to ask for anything that’s not in the devguide or not clear.

Thanks for the offer!
msg142600 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-21 09:05
> http://python.org/download/releases/
Ah, I understand: the release candidate for 3.2.2 is a testing release, but not a development one :)
msg142630 - (view) Author: Christian Ziemski (chriz) Date: 2011-08-21 16:29
Attached a patch against Python 3.3

All the existing tests are still running successfully.

But I wasn't able to add additional tests for the new functionality. 
Writing proper tests is a bit over my head for now - still learning.
msg142633 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-21 16:46
test_argparse has a proper test infrastructure so that people don’t have to write the same boilerplate and assertion code over and over again.  If you want, you can probably learn by copy-pasting things.

What you want to test is basically that shortened forms work and conflicting abbreviations report an error.
msg142733 - (view) Author: Christian Ziemski (chriz) Date: 2011-08-22 17:04
After some interesting hours of work (learning about tests) I created a new patch, including my original code changes as in the former patch and the new additional tests as well.
msg142768 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-22 22:41
Reviewed.  Steven: the patch is complete, pending a few doc editions.
msg149540 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2011-12-15 12:13
Modulo the comments already on the patch by others, this approach looks fine to me.
msg164077 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-06-26 13:56
You could consider just a small refinement: in the first loop in _SubParsersAction.__call__ where you look for the abbreviation, you can just set parser_name = p and break. Then the logic just below that can stay as it is: all you've done is morphed the abbreviation to the full command name.
msg183184 - (view) Author: Christian Ziemski (chriz) Date: 2013-02-27 20:26
Ouch, I really missed this one for a long time. :-(
(I didn't understand the workflow correctly and overlooked the reviews.)
I apologize to everyone who has been involved!

Finally I'm back here and re-did my patch for 3.4 this time.

I followed the comments of the reviewers (and Vinay's suggestion) as far as possible.
msg187240 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2013-04-18 11:52
The latest patch seems okay to me.  I've successfully applied it and ran the test suite and everything passed.  Could someone please take a look with a view to getting this committed, thanks.
msg273066 - (view) Author: Roundup Robot (python-dev) Date: 2016-08-18 20:24
New changeset 97b28115bd08 by Vinay Sajip in branch 'default':
Closes #12713: Allowed abbreviation of subcommands in argparse.
https://hg.python.org/cpython/rev/97b28115bd08
msg273151 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-08-19 18:36
There's a bug in this patch, first reported at https://github.com/python/benchmarks/issues/1

Attached patch causes test_parse_args_abbreviation to show the failure:

test.test_argparse.ArgumentParserError: ('SystemExit', '', "usage: PROG [-h] [--foo] bar {1,2,3,lost,long,longer} ...\nPROG: error: ambiguous choice: 'long' could match long, longer\n")
msg273168 - (view) Author: paul j3 (paul.j3) * Date: 2016-08-20 00:14
I haven't read the discussion in full, but it looks like this patch was added without any recent discussion or testing.

Previously if we had `choices=['a','abc']`, any exact match would be accepted, and partial matches rejected.  With this change the only choice that will work is the longest, 'abc'.  The other will be rejected as ambiguous.

In contrast with optionals flags, an exact match has priority over (possibly ambiguous) abbreviations.  See parser._parse_optional

I also noticed when testing this that for regular 'choices', the abbreviated string is put in the Namespace.  

     add_argument('--foo', choices = ['one','two'])

can produce `Namespace(foo='on')`.  I suspect most developers would want `Namespace(foo='one')`, regardless of whether it was an exact match or partial.  You don't list choices if you are ok with partial matches.

This isn't a problem with `subparsers`, since an abbreviation match still invokes the right subparser, and even puts the right name in 'subparsers' dest.  But for regular choices the behavior is highly debatable.

This needs to be reverted.  It may be fixable, but it needs more testing and discussion.  For now it's an enhancement that is causing backward compatibility problems.

ps

I missed this issue when I made an effort to find all argparse issues several years ago.  I contributed to https://bugs.python.org/issue14365, the other abbreviations issue mentioned in the recent github thread.
msg273319 - (view) Author: paul j3 (paul.j3) * Date: 2016-08-21 21:35
Another failure case:

parser.add_argument('--int', type=int, choices=[10,15,25])

'--int 15` puts 'int=15' in the Namespace.

'--int 2' puts 'int=2' in the Namespace, because it matches the 'str(25)'.

'--int 1' raises an error in the ambiguity error formatting code: 

     'choices': ', '.join(ac)}

This last error could be corrected with:

    'choices': ', '.join(map(repr, ac))

=================

The initial message for this issue stated:

    So that this feature is not foisted on every programmer that uses  
    argparse it could be made an option to the ArgumentParser class with 
    the (possible) additional refinement of specifying a minimum number 
    of characters for the abbreviations.

That was ignored in the subsequent discussion and patches.

Other Choices issues

http://bugs.python.org/issue16468 - argparse only supports iterable choices
http://bugs.python.org/issue16418 - long choices
http://bugs.python.org/issue16977 - choices='abc' case

I recommend closing this issue, and depend on aliases for subparsers abbreviations.  The interaction with 'choices' is just too complicated to handle as proposed here.
msg273394 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2016-08-22 18:36
Mean culpa - I have  jumped the gun on this. Sorry to all for the inconvenience. I have just got back after being away for a few days, and will revert the change shortly, if no one beats me to it.
msg273395 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2016-08-22 18:36
*Mea culpa. Autocorrect :-(
msg273420 - (view) Author: Roundup Robot (python-dev) Date: 2016-08-23 07:43
New changeset 3310ea7dbe30 by Vinay Sajip in branch 'default':
Issue #12713: reverted fix pending further discussion.
https://hg.python.org/cpython/rev/3310ea7dbe30
History
Date User Action Args
2016-08-23 07:54:28vinay.sajipsetassignee: vinay.sajip ->
2016-08-23 07:48:01vinay.sajipsetpriority: release blocker -> normal
2016-08-23 07:43:31python-devsetmessages: + msg273420
2016-08-22 18:36:55vinay.sajipsetmessages: + msg273395
2016-08-22 18:36:23vinay.sajipsetmessages: + msg273394
2016-08-21 21:35:29paul.j3setmessages: + msg273319
2016-08-20 01:46:25ned.deilysetpriority: normal -> release blocker
2016-08-20 00:14:25paul.j3setkeywords: + needs review, - patch

messages: + msg273168
2016-08-19 21:57:37paul.j3setnosy: + paul.j3
2016-08-19 18:36:01zach.waresetstatus: closed -> open
files: + abbr_bug_test.diff

assignee: vinay.sajip
versions: + Python 3.6, - Python 3.4
nosy: + zach.ware

messages: + msg273151
resolution: fixed ->
stage: resolved -> needs patch
2016-08-18 20:24:24python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg273066

resolution: fixed
stage: test needed -> resolved
2014-02-03 18:33:07BreamoreBoysetnosy: - BreamoreBoy
2013-04-18 11:52:49BreamoreBoysetnosy: + BreamoreBoy
messages: + msg187240
2013-02-27 20:26:01chrizsetfiles: + abbrev_subcmds_incl_tests_3.4.patch

messages: + msg183184
versions: + Python 3.4, - Python 3.3
2012-06-26 13:56:31vinay.sajipsetnosy: + vinay.sajip
messages: + msg164077
2012-03-19 01:39:04bethardunlinkissue14365 superseder
2012-03-18 23:16:57bethardlinkissue14365 superseder
2012-03-18 21:45:34tshepangsetnosy: + tshepang
2011-12-15 12:13:49bethardsetmessages: + msg149540
2011-08-22 22:41:25eric.araujosetmessages: + msg142768
2011-08-22 17:04:34chrizsetfiles: + abbrev_subcmds_plus_tests_3.3.patch

messages: + msg142733
2011-08-21 16:46:16eric.araujosetmessages: + msg142633
stage: test needed
2011-08-21 16:29:29chrizsetfiles: + abbrev_subcmds_3.3.patch

messages: + msg142630
2011-08-21 09:05:50eric.araujosetmessages: + msg142600
2011-08-21 09:03:40chrizsetmessages: + msg142599
2011-08-21 08:18:26eric.araujosetmessages: + msg142598
2011-08-21 06:08:35chrizsetmessages: + msg142592
2011-08-20 21:20:48eric.araujosetmessages: + msg142567
2011-08-20 09:02:39chrizsetmessages: + msg142519
2011-08-20 08:49:56chrizsetfiles: + abbrev_subcmds.patch
keywords: + patch
messages: + msg142518
2011-08-20 05:40:30ned.deilysetnosy: + ned.deily
messages: + msg142513
2011-08-19 19:30:30chrizsetnosy: + chriz
messages: + msg142492
2011-08-12 16:20:08eric.araujosetnosy: + eric.araujo, bethard

messages: + msg141968
versions: + Python 3.3
2011-08-09 01:55:56pwil3058create