classification
Title: Issue with spacing in argparse module while using help
Type: Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: cheryl.sabella, falu2010, paul.j3, r.david.murray, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-02-22 23:00 by falu2010, last changed 2017-05-28 23:25 by paul.j3.

Pull Requests
URL Status Linked Edit
PR 1835 closed cheryl.sabella, 2017-05-27 19:18
Messages (11)
msg288389 - (view) Author: Falguniben Jhaveri (falu2010) Date: 2017-02-22 23:00
When I use argparse the optional arguments with character "-p" and "-s" has improper spacing while using help/usage command.

Sample out put from argparse module:

usage: cli node list [-h] [-p] [-o]

Lists nodes in your current project.

optional arguments:
  -h, --help         show this help message and exit
  // extra space after p
  -p , --projectid  
  -o, --org         (For administrators only) Lists all the nodes in
msg288403 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2017-02-23 04:43
We need to see the parser setup as well.  I've never seen a bug like this before.  The `usage` line suggests that you are using subparsers.

It might be better if you asked this on StackOverFlow with a repeatable code example.  That's a better place to get debugging help of your own code.  Come back here if others think this is a problem with `argparse` itself rather than your own setup.
msg288674 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2017-02-27 21:57
With this setup

import argparse
parser=argparse.ArgumentParser(prog='cli')
parser.add_argument('nodes')
sp=parser.add_subparsers()
p1 = sp.add_parser('list', description='Lists nodes in your current project')
p1.add_argument('-p','--projectid', action='store_true')
p1.add_argument('-o', '--org', action='store_true', help=' (For administrators only) Lists all the nodes in')
parser.parse_args()


this help looks normal

1354:~/mypy/argdev$ python3 issue29626.py nodes list -h
usage: cli nodes list [-h] [-p] [-o]

Lists nodes in your current project

optional arguments:
  -h, --help       show this help message and exit
  -p, --projectid
  -o, --org        (For administrators only) Lists all the nodes in

Without further feedback this issue should be closed
msg288676 - (view) Author: Falguniben Jhaveri (falu2010) Date: 2017-02-27 22:07
Sample code to repro this:

import argparse

    parser = argparse.ArgumentParser(prog='cli')
    subparsers = parser.add_subparsers(dest="command")

    subparsers_user_delete = subparsers.add_parser("delete", help="Deletes a user in your organization.",
                                                        description="Deletes a user in your organization.")
    subparsers_user_delete.add_argument("userid", help="The userid of user.", default=None, type=int)
    subparsers_user_delete.add_argument("-p", "--projectid", metavar="",
                                        help="Specify the project ID of project from where you want to delete"
                                             " the user or else user will be deleted from organization.", type=int,
                                        default=None)

    parser_nodes = subparsers.add_parser('nodes')
    sp = parser_nodes.add_subparsers()
    p1 = sp.add_parser('list', description='Lists nodes in your current project')
    p1.add_argument('-p', '--projectid', action='store_true')
    p1.add_argument('-o', '--org', action='store_true', help=' (For administrators only) Lists all the nodes in')
    parser.parse_args()

Please run following command:
$issue.py delete -h
msg288679 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2017-02-27 22:39
This help looks normal:

1427:~/mypy/argdev$ python3 issue29626.py delete -h
usage: cli delete [-h] [-p] userid

Deletes a user in your organization.

positional arguments:
  userid             The userid of user.

optional arguments:
  -h, --help         show this help message and exit
  -p , --projectid   Specify the project ID of project from where you want to
                     delete the user or else user will be deleted from
                     organization.

So does this help for the nested subparser:

1430:~/mypy/argdev$ python3 issue29626.py nodes list -h
usage: cli nodes list [-h] [-p] [-o]

Lists nodes in your current project

optional arguments:
  -h, --help       show this help message and exit
  -p, --projectid
  -o, --org        (For administrators only) Lists all the nodes in

This double layered subparsers is not common, and might not even be included in the unittest file.  But provided you don't try anything too tricky it does work.  I've seen a few questions along this line on StackOverflow.

Note that the help line for '-p' in the second case is empty because you did not specify any help string (as you did for 'delete').
msg288681 - (view) Author: Falguniben Jhaveri (falu2010) Date: 2017-02-27 22:44
There is an extra spacing here if you look carefully. I have commented
above the code. Similarly it happens for "-o" too but not for other
alphabets.

1427:~/mypy/argdev$ python3 issue29626.py delete -h
usage: cli delete [-h] [-p] userid

Deletes a user in your organization.

positional arguments:
  userid             The userid of user.

optional arguments:
  -h, --help         show this help message and exit
// extra space after p and ",".
  -p , --projectid   Specify the project ID of project from where you want
to
                     delete the user or else user will be deleted from
                     organization.

I saw this issue as I use argparse for my CLI usecase which has nested
commands.

On Mon, Feb 27, 2017 at 2:39 PM, paul j3 <report@bugs.python.org> wrote:

>
> paul j3 added the comment:
>
> This help looks normal:
>
> 1427:~/mypy/argdev$ python3 issue29626.py delete -h
> usage: cli delete [-h] [-p] userid
>
> Deletes a user in your organization.
>
> positional arguments:
>   userid             The userid of user.
>
> optional arguments:
>   -h, --help         show this help message and exit
>   -p , --projectid   Specify the project ID of project from where you want
> to
>                      delete the user or else user will be deleted from
>                      organization.
>
> So does this help for the nested subparser:
>
> 1430:~/mypy/argdev$ python3 issue29626.py nodes list -h
> usage: cli nodes list [-h] [-p] [-o]
>
> Lists nodes in your current project
>
> optional arguments:
>   -h, --help       show this help message and exit
>   -p, --projectid
>   -o, --org        (For administrators only) Lists all the nodes in
>
> This double layered subparsers is not common, and might not even be
> included in the unittest file.  But provided you don't try anything too
> tricky it does work.  I've seen a few questions along this line on
> StackOverflow.
>
> Note that the help line for '-p' in the second case is empty because you
> did not specify any help string (as you did for 'delete').
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue29626>
> _______________________________________
>
msg288683 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2017-02-27 23:48
Sorry, I missed that.  For some reason I looking something bigger.

That's coming from the `metavar=""'.

If I specify `metavar="xxx" that help line will have

    -p xxx, --projectid xxx

Replace the 'xxx` with '', and you still have space between '-p' and ','.

Now that I see it, it looks familiar.  I noted it in passing in StackOverflow answer, http://stackoverflow.com/a/40497623/901925

I can't find a related bug/issue.  

It's a natural consequence of the formatting in HelpFormatter._format_action_invocation

            # if the Optional takes a value, format is:
            #    -s ARGS, --long ARGS
            parts.append('%s %s' % (option_string, args_string))

There's no special handling for the case where ARGS is blank.

That formatter method could be customized as suggested in 

http://stackoverflow.com/a/23941599/901925

Often people want a more compact invocation like:

  -s, --long ARG    help

Usage gets that space between option_string and args_string, but it gets striped out later.

So the fix (not tested) would something like:

      def _format_action_invocation(self, action):
           ....
           for option_string in action.option_strings:
               if len(args_string)>0:
                    parts.append('%s %s' % (option_string, args_string))
               else:
                    parts.append('%s' % option_string)
           ....
msg294615 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-27 22:18
Does an empty metavar make sense? This makes the option with argument non-distinguising from the option without argument.
msg294619 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2017-05-28 01:05
I submitted a PR for this based on paul.j3's suggested change:

      def _format_action_invocation(self, action):
           ....
           for option_string in action.option_strings:
               if len(args_string)>0:
                    parts.append('%s %s' % (option_string, args_string))
               else:
                    parts.append('%s' % option_string)
           ....

I created a test that would have duplicated the original issue and, with the change, no longer prints the extra blank.

To Serhiy's point, I'm not sure if an empty metavar makes sense since it's intention is to give the parser a way to refer to the expected arguments within the help messages. 

However, with no metavar at all, the help output is:
  -f FOO, --foo FOO

With metavar = '' (and this change), the output is:
-f, --foo 

With metavar = '' (and without this change), the output is:
-f , --foo
msg294655 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-05-28 19:14
If I'm interpreted what the OP wrote correctly, he wanted the help text to not show that the option takes an argument, but instead rely on the help text to show that.  That works for the option text, but it doesn't work for the synopsis (the synopsis is just wrong in that case).

Probably a blank metavar should just be rejected because of the issue with they synopsis.  That should only be done in 3.7 if we do it, though.

Perhaps the OP will respond to explain the blank-metavar use case in more detail.
msg294663 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2017-05-28 23:25
I don't anticipate  any backward compatibility issues with the proposed patch.  But at the same time it feels almost too trivial of an issue to merit a patch.  

A short metavar such as `metavar="ID"' or even `metavar="_"' would be nearly as compact, and still leave an indicator of the required argument (in usage as well as the help).

I'm not even sure if disallowing a blank metavar is worth the change.

As indicated before I've only seen this extra space problem when I or others suggest the blank metavar as a way making a more compact help invocation.  That has come up frequently on SO, but I can't find a relevant bug/issue.  

https://stackoverflow.com/a/18280876/901925 - here I suggest suppressing the help line and putting custom info in the group's description.

https://stackoverflow.com/questions/9642692/argparse-help-without-duplicate-allcaps - suggests a help formatter subclass

So maybe the bigger issue is, can we make it easier to customize the the help invocation formatting?

Things to consider:

- metavar affects both the help line and usage

- setting help width and indentation is a messy part of the formatter

- there are unittests for long and long-long-longer option names to test these indentation issues.

As with some other help formatter issues, we are treading a fine line between making trivial patches, and substantive ones that make it cleaner and more robust.
History
Date User Action Args
2017-05-28 23:25:22paul.j3setmessages: + msg294663
2017-05-28 19:14:19r.david.murraysetmessages: + msg294655
2017-05-28 01:05:58cheryl.sabellasetnosy: + cheryl.sabella
messages: + msg294619
2017-05-27 22:18:58serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg294615
2017-05-27 21:17:36r.david.murraysetnosy: + r.david.murray
2017-05-27 19:18:36cheryl.sabellasetpull_requests: + pull_request1918
2017-02-27 23:48:43paul.j3setstatus: closed -> open
2017-02-27 23:48:31paul.j3setmessages: + msg288683
2017-02-27 22:44:31falu2010setmessages: + msg288681
2017-02-27 22:39:21paul.j3setmessages: + msg288679
2017-02-27 22:07:31falu2010setmessages: + msg288676
2017-02-27 21:57:25paul.j3setstatus: open -> closed
stage: resolved
2017-02-27 21:57:15paul.j3setmessages: + msg288674
2017-02-23 04:43:24paul.j3setnosy: + paul.j3
messages: + msg288403
2017-02-22 23:00:55falu2010create