The argparse module lists required args as optional in the default help message.

If you run the following program (also attached) you'll get the output listed below.

#!/usr/bin/env python

import argparse

parser = argparse.ArgumentParser(
    description = 'Do something'
)
parser.add_argument('--reqarg', '-r', help = 'This is required', required = True)
parser.add_argument('--optarg','-o', help = "This is optional", required = False)
args = parser.parse_args()

$ python argparse-help-says-required-args-are-optional.py -h
usage: argparse-help-says-required-args-are-optional.py [-h] --reqarg REQARG
                                                        [--optarg OPTARG]

Do something

optional arguments:
-h, --help show this help message and exit
--reqarg REQARG, -r REQARG
This is required
--optarg OPTARG, -o OPTARG
This is optional
$

It looks to me like reqarg is marked as required, since it's not in brackets. Or am I missing something?

Yeah, the fact that it is listed under the heading "optional arguments:" :) Guess we need a new section?

Duh. Sorry about that.

Also applies to 3.2.

Yeah, I guess the optional vs. positional isn't the best terminology now that you can have required flag-based arguments. Did you have a word other than "optional" that you'd prefer?

Perhaps you could just label them 'options:'? After all, even if you have several options you may be required to pick at least one :)

Or "parameters:"?

FWIW, I like the idea of just using the label "options".

If you add a positional parameter by adding:
parser.add_argument('foo')
then the output becomes:

$ python argparse-help-says-required-args-are-optional.py -h
usage: issue9649.py [-h] --reqarg REQARG [--optarg OPTARG] foo

Do something

positional arguments:
foo

optional arguments:
-h, --help show this help message and exit
--reqarg REQARG, -r REQARG
This is required
--optarg OPTARG, -o OPTARG
This is optional

$

So whatever replaces "optional arguments:" needs to read well with "positional arguments:". Maybe just plain "options:" is good enough, but I think a word to replace "optional" (leaving "arguments:") would be better. I just don't have any useful suggestion :)

I guess one possibility might be "flag arguments". It's not great, but I guess it's more accurate.

And I guess the bigger issue to think about is how to add this in a backwards compatible way. I guess we could just add methods like "set_positionals_group_name(name)" and then fiddle with "self._positionals.title" in there. Not sure that's a great solution though - it seems like adding one method to change just this single attribute is overkill and not very general.

In the meantime, here's a workaround:

>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo', required=True)
>>> parser._optionals.title = "flag arguments"
>>> parser.print_help()
usage: PROG [-h] --foo FOO

flag arguments:
-h, --help show this help message and exit
--foo FOO

I can't promise this will continue to work, since it uses the undocumented _optionals attribute, but at least it's a way of getting something like what you want now.

Well, there's also bpo-9652, which speaks to having a more general facility, I suppose. Maybe an exposed dictionary attribute containing the constant strings?

Is this really a behavior bug or doc bug?
Or a feature request for better message customization?

*This* bug is a behavior bug (required flags are mis-labelled as being optional in the help text). The referenced bug is a feature request, but it may make sense to consider it while fixing this one.

I think this is still really a feature request. We can't just change the text from "optional" - that would silently change a large number of help messages without any warning. So to fix this "bug", we're going to have to add an API to explicitly set the group names - which can only be done as a new feature. People using 2.7 will have to use the workaround using parser._optionals I posted here.

So it strikes me that there already exists an officially supported way to rename your option groups. Just only create your own option groups (never use the default ones) and only put arguments there, e.g.:

------------------------- temp.py --------------------------

parser = argparse.ArgumentParser(description = 'Do something', add_help=False)
flags = parser.add_argument_group('flag arguments')
flags.add_argument('-h', '--help', action='help')
flags.add_argument('--reqarg', '-r', help='This is required', required=True)
flags.add_argument('--optarg','-o', help="This is optional", required=False)
args = parser.parse_args()

$ python temp.py --help
usage: temp.py [-h] --reqarg REQARG [--optarg OPTARG]

Do something

flag arguments:
-h, --help
--reqarg REQARG, -r REQARG
This is required
--optarg OPTARG, -o OPTARG
This is optional
------------------------------------------------------------

The documentation for action='help' needs to be added, as pointed out in Issue# 10772.

So basically, the API for customizing group names is already there. So I'm changing this to a documentation request - there should be an example in the docs showing how to change the default group names as above.

I'm changing the title because I keep seeing duplicates.

Documentation patches still welcome!

How about calling required arguments "required arguments"?

required arguments:
--reqarg REQARG, -r REQARG
This is required
optional arguments:
-h, --help show this help message and exit
--optarg OPTARG, -o OPTARG
This is optional

Clear and unambiguous. With this approach the user does not have to bloat the help to state "This is required".

We're having the same discussion over at github regarding argparse4j:
argparse4j/argparse4j#26 (comment)

A new “required arguments” section seems too arbitrary to me. It would clash with the “positional arguments” heading, since those are also required by default.

I would go with the heading “options”, as a noun. That term seems to be well used, at least on Linux and Wikipedia (see https://en.wikipedia.org/wiki/Command-line_option). Other terms are “flag” and “switch”. In this thread I see two arguments against this:

1. Eric Smith prefers to retain the noun “arguments”. How about something like “non-positional arguments” then?

2. Steven Bethard is worried about backwards compatibility. I thought the Python people were happy to make these sort of changes each minor release (e.g. 3.4 to 3.5).

The module’s source code uses the term “optionals” a lot more than this one heading. It would be clearer if this term were dropped, or only used for things that are truly optional. So even if you can’t fix the help output until Python 4, please fix the documentation and the rest of the source code :)

How about calling required arguments "required arguments"?
...
Clear and unambiguous. With this approach the user does
not have to bloat the help to state "This is required".

+1 This is straight-forward, logical, and easy-to-read.

As Steven pointed out, the existing add_argument_group mechanism can be used to group required arguments. For example

------------------------- temp.py \--------------------------

    parser = argparse.ArgumentParser(description = 'Do something')
    group1 = parser.add_argument_group('required arguments')
    group1.add_argument('--reqarg', '-r', required=True)
    parser.add_argument('--optarg','-o')
    parser.add_argument('foo')
    parser.print_help()
    

usage: ipython [-h] --reqarg REQARG [--optarg OPTARG] foo
Do something
positional arguments:
  foo
optional arguments:
  -h, --help            show this help message and exit
  --optarg OPTARG, -o OPTARG
required arguments:
  --reqarg REQARG, -r REQARG

Positional 'foo' can also be put in the 'required' group:

group1.add_argument('foo')

required arguments:
  --reqarg REQARG, -r REQARG
  foo

The distinction between 'positionals' and 'optionals' (or flagged) is essential to the parsing, but it is not necessary for Help Formatting.

I can imagine grouping arguments by 'required/not-required' properties. It might be worth constructing an alternative HelpFormatter class that regroups the arguments in this way. Subclassing the HelpFormatter is the established way of adding features to the help display.

The existing HelpFormatter flags 'required' arguments in the usage line with '[]'. There it is has the added task of flagging Mutually Exclusive Groups in the same way.

It's worth keeping in mind that whether an argument is 'required' or not is determined in 2 different ways. There is an optional 'required' flag (default False). But this flag is not allowed for 'positionals'. Instead with those 'argparse' looks at 'nargs' ('?*' are not required).

The 'required' attribute of an argument (Action) is ignored during 'parse_args' until the end. At that time it makes an inventory of 'required' arguments that have not been seen, and potentially raises an error. That testing was changed in a relatively recent patch, and produced an unintended change in whether 'subparsers' were required or not. (I could look up those issues in needed).

I'll think about creating the alternative HelpFormatter.

The attached file shows how the default argument groups could be redefined, using 'required' as the criteria.

I've implemented it as a method that is added to a subclass of ArgumentParser. This method is invoked after arguments are defined, prior to generating the help.

The help looks something like this:

usage: alt_grouping.py [-h] [-f FOO] -g GOO pos [baz]

required:
  pos                required positional
  -g GOO, --goo GOO  required optional

optional:
  -h, --help         show this help message and exit
  -f FOO, --foo FOO  optional
  baz                optional positional

I was thinking of implementing this as a formatter subclass, but given the way the help is assembled, invoking this method from the parser is simpler.

Is there anything that can be done to help this issue move along? I just ran into it just now

Mark,

Have you tried defining your own Argument Group? If that didn't work, what fix do you want? Why?

I have just run into the same issue here: https://bugs.python.org/issue38950

• I prefer Terry J. Reedy's "keyword arguments" as it is clear and consistent with "positional arguments".
• But Steven Bethard 's "flag arguments" looks fine since it is well known to shell users.
• Martin Panter's "options" looks okay since it is the standard name in GNU Coreutils (https://www.gnu.org/software/coreutils/manual/coreutils.html#Common-options). However I don't like it very much as it is still ambiguous: "options", like "optional arguments", still suggests something that is non required. And "options" is less consistent with "positional arguments" (nobody seems to have suggested "option arguments").
• Oliver Smith's "switches" does not look okay because it is not general enough since it is commonly restricted to Boolean arguments.

Anyway, the first 3 solutions are better than the current "optional arguments". What is blocking the approval of Martin Panter's PR?

It seems the discussion has so far revolved around 'optional' arguments with required=True.

What about the other way around?

While trying to set required=False for a positional argument raises an exception, it is still possible to make the positional argument effectively optional by nargs='?' (it is then printed in brackets in usage message).

Based on the attached survey of practices, I propose a minimal edit to the help display. Instead of "optional arguments:", we say "the options are as follows:".

The use of the word "option" is dominant is in the CLI world, followed by "action" and "switch". The noun form "option" doesn't seem to cause the same confusion that arises in the adjective form "optional arguments" which strongly implies "not required".

For the documentation, I suggest adding a sentence or two in the introduction to explain the terminology used throughout the rest of the argparse docs.

Since any chance to the help output will likely break tests, marking this as 3.10 only.

I also like Eric'c suggestion of just using "options:" instead of "optional arguments".

Since this change will break tests that rely matching help output exactly, I would like to hear if there are any objections to replacing "optional arguments" with "options".

The words "switch" or "flag" don't work as well because they imply on/off and don't encompass option that take arguments.

I wouldn't let breaking these tests deter you from improving the output. I think using "options" is an improvement.

New changeset 41b223d by Raymond Hettinger in branch 'master':
bpo-9694: Fix misleading phrase "optional arguments" (GH-23858)
41b223d

Coudl this please be mentioned on https://docs.python.org/3.10/whatsnew/3.10.html ?

At least two packages fail tests because of the change (ipython and sphinxcontrib-autoprogram).

Since this issue is closed it might be a good idea to open a new one with this problem. And if possible identify the failed tests.

We forgot to allow for the fact that working code/tests might be checking for specific help messages, checks the will fail when this group label is changed.

https://bugs.python.org/issue42870