classification
Title: argparse subcommands not printed in the same order they were added
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: bethard Nosy List: bethard, ddvento@ucar.edu, eric.araujo, jcollado, ned.deily, python-dev, r.david.murray
Priority: normal Keywords: patch

Created on 2010-06-18 10:24 by jcollado, last changed 2015-04-02 18:05 by ddvento@ucar.edu. This issue is now closed.

Files
File name Uploaded Description Edit
subcommands.py jcollado, 2010-06-18 10:24 Subcommands example to reproduce the problem
trunk.diff jcollado, 2010-06-21 15:27 Patch for trunk branch
py3k.diff jcollado, 2010-06-21 15:32 Patch for the py3k branch
pre27.diff jcollado, 2010-06-21 16:18 Patch for python < 2.7
Messages (20)
msg108096 - (view) Author: Javier Collado (jcollado) Date: 2010-06-18 10:24
What steps will reproduce the problem?
1. Run 'python subcommands.py -h' (attached file)
2. Check the ordering of the subcommands in the output:
----
subcommands:
  {a,c,b,e,d}
    a          a subcommand help
    b          b subcommand help
    c          c subcommand help
    d          d subcommand help
    e          e subcommand help
----

The ordering between brackets is different than the ordering in the one-line help for each command below. This could be a little confusing to the user.


What is the expected output? What do you see instead?
The expected output would be to use the same ordering in both places. In particular the same ordering that was used in the code when using the _SubParsersAction.add_parser method:
----
subcommands:
  {a,b,c,d,e}
    a          a subcommand help
    b          b subcommand help
    c          c subcommand help
    d          d subcommand help
    e          e subcommand help
----


What version of the product are you using? On what operating system?
OS: Ubuntu
Version: source code (rev. 87)
Python: 2.6.5

Please provide any additional information below.
Please have a look at the attached diff. It contains a patch that worked for me to preserve the ordering used in the code.

To make that possible it uses action._choices_actions (a list) instead of action.choices (a dictionary).
msg108098 - (view) Author: Javier Collado (jcollado) Date: 2010-06-18 10:26
It contains a patch that worked for me to preserve the ordering used in the code.

To make that possible it uses action._choices_actions (a list) instead of action.choices (a dictionary).
msg108110 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-06-18 14:00
It would be simpler to use an OrderedDict.
msg108112 - (view) Author: Javier Collado (jcollado) Date: 2010-06-18 15:07
While the ordered dict is a nice option, the one-line patch that is attached to the report works in python < 2.7 without adding any external dependency.

In my opininion, that's interesting for those using argparse with earlier versions of python.
msg108113 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-06-18 15:14
Sorry, "simpler" was the wrong choice of word :)

Anyway, it's up to Steven.
msg108167 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-06-19 04:57
Thanks for the nice report and patch! I don’t know if Steven maintains argparse first in Python’s repository and then makes external releases or if he touches first his argparse repo and then Python’s, but if it’s the former, please generate your diff again from the top directory of the checkout (see http://www.python.org/dev/patches/).

Could you add tests too?
msg108209 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-06-19 20:42
Yes, please generate patches from the Python repository.

Thanks!
msg108210 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-06-19 20:46
Guidelines: http://www.python.org/dev/patches/
msg108254 - (view) Author: Javier Collado (jcollado) Date: 2010-06-21 06:59
Working on it.
msg108275 - (view) Author: Javier Collado (jcollado) Date: 2010-06-21 15:27
Finally I had to use an OrderedDict as suggested by R. David Murray because it wasn't safe to rely on _choices_actions in HelpFormatter class (i.e. previous patch wasn't valid):
- _choices_actions attribute is only present in _SubParsersAction class
- Even if action object is an instance of _SubParsersAction, _choices_actions only contains data for for subparsers that contain a help description.

Regarding the test cases:
- TestHelpSubparsersOrdering and TestHelpSubparsersWithHelpOrdering have been added
- TestHelpFormattingMetaClass has been modified:
  - New subparsers_signatures tester attribute added to test subparsers help.
  - If a 'signatures attribute' isn't present in tester object, then isn't consumed
  - assertMultilineEqual used instead of assertEqual to make it easier to debug test case failures.
msg108277 - (view) Author: Javier Collado (jcollado) Date: 2010-06-21 15:32
Despite trunk.diff can be successfully applied to py3k branch, please find attached the patch generated from py3k branch.
msg108284 - (view) Author: Javier Collado (jcollado) Date: 2010-06-21 16:18
Just for the record, please find attached an uglier patch for python < 2.7 that doesn't have the same problems as the first one I uploaded and passes all the test cases (tested with python 2.6).
msg110616 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-07-17 23:23
In py3k.diff, the changes to argparse and new test cases look good to me. I don’t know why you’ve added hasattr tests in test_argparse.py.
msg111334 - (view) Author: Javier Collado (jcollado) Date: 2010-07-23 14:33
The hasattr expressions were added to TestHelpFormattingMetaclass because:
- A new attribute (subparsers_signature) was added in test classes that wasn't used in the past.
- The new test classes didn't make use of some of the already in place attributes (argument_signatures, argument_signatures)
- The old test classes didn't make use of the new attribute (subparsers_signature).
- TestHelpFormattingMetaclass assumed that those attributes were present in test classes (otherwise an exception was raised).

Hence, in order not to change/break a lot of test cases, I decided to add the hasattr expressions.
msg132315 - (view) Author: Roundup Robot (python-dev) Date: 2011-03-27 12:04
New changeset 74f9ed48ae5d by Steven Bethard in branch '3.2':
Issue #9026: Fix order of argparse sub-commands in help messages.
http://hg.python.org/cpython/rev/74f9ed48ae5d

New changeset de29472c6a84 by Steven Bethard in branch 'default':
Issue #9026: Fix order of argparse sub-commands in help messages. (Merged from 3.2.)
http://hg.python.org/cpython/rev/de29472c6a84
msg132316 - (view) Author: Roundup Robot (python-dev) Date: 2011-03-27 12:04
New changeset 75ec20b4c50e by Steven Bethard in branch '2.7':
Issue #9026: Fix order of argparse sub-commands in help messages. (Merged from 3.2.)
http://hg.python.org/cpython/rev/75ec20b4c50e
msg132317 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2011-03-27 12:06
Sorry for letting this bug sit around for so long. I committed a slight variant of your patch to 2.7, 3.2 and 3.3. Thanks!
msg239853 - (view) Author: (ddvento@ucar.edu) Date: 2015-04-01 22:16
This problem is occurring again in python 2.7.7, can we open it again?
msg239884 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-04-02 07:45
@ddvento: This issue has been closed and the fixes for it released several years ago.  Comments added here will likely be ignored.  If you believe there is a problem with current releases (for Python 2, Python 2.7.9 is current), please open a new issue and document how to reproduce, including on what platform(s) and with which versions of Python.  FWIW, the original test case in this issue works correctly with both a 2.7.6 and current 2.7.9 on the platforms I tried it with.
msg239937 - (view) Author: (ddvento@ucar.edu) Date: 2015-04-02 18:05
You are right, this problem is not coming from python itself, but more from
setuptools and its use by scoop. See
https://github.com/soravux/scoop/issues/16 and
http://stackoverflow.com/questions/29374044/ for details

Regards,
Davide Del Vento,
NCAR Computational & Information Services Laboratory
Consulting Services Software Engineer
http://www2.cisl.ucar.edu/uss/csg/
SEA Chair http://sea.ucar.edu/
office: Mesa Lab, Room 55G
phone:  (303) 497-1233

On Thu, Apr 2, 2015 at 1:45 AM, Ned Deily <report@bugs.python.org> wrote:

>
> Ned Deily added the comment:
>
> @ddvento: This issue has been closed and the fixes for it released several
> years ago.  Comments added here will likely be ignored.  If you believe
> there is a problem with current releases (for Python 2, Python 2.7.9 is
> current), please open a new issue and document how to reproduce, including
> on what platform(s) and with which versions of Python.  FWIW, the original
> test case in this issue works correctly with both a 2.7.6 and current 2.7.9
> on the platforms I tried it with.
>
> ----------
> nosy: +ned.deily
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue9026>
> _______________________________________
>
History
Date User Action Args
2015-04-02 18:05:09ddvento@ucar.edusetmessages: + msg239937
2015-04-02 07:45:24ned.deilysetnosy: + ned.deily
messages: + msg239884
2015-04-01 22:16:19ddvento@ucar.edusetnosy: + ddvento@ucar.edu
messages: + msg239853
2011-03-27 12:06:28bethardsetstatus: open -> closed
versions: + Python 3.3
messages: + msg132317

resolution: fixed
stage: patch review -> resolved
2011-03-27 12:04:52python-devsetmessages: + msg132316
2011-03-27 12:04:19python-devsetnosy: + python-dev
messages: + msg132315
2010-08-10 09:49:32bethardlinkissue9537 superseder
2010-07-23 14:33:29jcolladosetmessages: + msg111334
2010-07-17 23:23:05eric.araujosettitle: [argparse] Subcommands not printed in the same order they were added -> argparse subcommands not printed in the same order they were added
messages: + msg110616
stage: test needed -> patch review
2010-06-21 16:18:39jcolladosetfiles: + pre27.diff

messages: + msg108284
2010-06-21 15:32:06jcolladosetfiles: + py3k.diff

messages: + msg108277
2010-06-21 15:28:20jcolladosetfiles: - ordered_subcommands.diff
2010-06-21 15:28:01jcolladosetfiles: + trunk.diff

messages: + msg108275
2010-06-21 06:59:18jcolladosetmessages: + msg108254
2010-06-19 20:46:01eric.araujosetmessages: + msg108210
2010-06-19 20:42:35bethardsetmessages: + msg108209
2010-06-19 04:58:00eric.araujosetnosy: + eric.araujo

messages: + msg108167
stage: test needed
2010-06-18 15:14:29r.david.murraysetmessages: + msg108113
2010-06-18 15:07:20jcolladosetmessages: + msg108112
2010-06-18 14:00:57r.david.murraysetnosy: + r.david.murray
messages: + msg108110
2010-06-18 11:32:55bethardsetassignee: bethard

nosy: + bethard
2010-06-18 10:26:25jcolladosetfiles: + ordered_subcommands.diff
keywords: + patch
messages: + msg108098
2010-06-18 10:24:37jcolladocreate