Title: argparse help formatter crashes
Type: behavior Stage:
Components: Library (Lib) Versions: Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Endre, berker.peksag, bethard, paul.j3, xiang.zhang
Priority: normal Keywords:

Created on 2016-05-04 15:14 by Endre, last changed 2018-07-11 07:43 by serhiy.storchaka.

Messages (6)
msg264823 - (view) Author: Endre (Endre) Date: 2016-05-04 15:14
I am using argparse-1.4.0.

For this code exception is thrown when the script is started with the -h option:

This exception is thrown:
Traceback (most recent call last):
  File "C:\Users\ebak\Picea\tools\buildsystem\python\", line 129, in <module>
    args = parser.parse_args()
  File "C:\Python27\lib\", line 1688, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "C:\Python27\lib\", line 1720, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "C:\Python27\lib\", line 1926, in _parse_known_args
    start_index = consume_optional(start_index)
  File "C:\Python27\lib\", line 1866, in consume_optional
    take_action(action, args, option_string)
  File "C:\Python27\lib\", line 1794, in take_action
    action(self, namespace, argument_values, option_string)
  File "C:\Python27\lib\", line 994, in __call__
  File "C:\Python27\lib\", line 2327, in print_help
    self._print_message(self.format_help(), file)
  File "C:\Python27\lib\", line 2301, in format_help
    return formatter.format_help()
  File "C:\Python27\lib\", line 279, in format_help
    help = self._root_section.format_help()
  File "C:\Python27\lib\", line 209, in format_help
  File "C:\Python27\lib\", line 317, in _format_usage
    action_usage = format(optionals + positionals, groups)
  File "C:\Python27\lib\", line 388, in _format_actions_usage
    start = actions.index(group._group_actions[0])
IndexError: list index out of range
msg264834 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-05-04 16:14
I don't think add_mutually_exclusive_group and add_argument_group are designed to work together. Did this worked with argparse 1.3.0 or stdlib version of argparse before? I'm getting the same traceback in 2.7 and 3.4+.
msg264835 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-05-04 16:15
This error occurs due to there is no action belong to the mutexGroup, the code requires at least one. You can simply add

    mutexGroup.add_argument('--foo', action='store_true')

and you'll see the error disappears. I have no idea that this behaviour is intended or not.
msg264837 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-05-04 16:23
I agree with Berker. Even without any exceptions, the help message given is not what add_argument_group and add_mutually_exclusive_group intend.

You can see

usage: [-h] [--foo] -u URL -p PROJECT [--dump] --mergeInput
                        MERGEINPUT [--removeDisabled] -c CHECKERS

Sets the checkers of a klocwork project

optional arguments:
  -h, --help            show this help message and exit
  -c CHECKERS, --checkers CHECKERS
                        File which lists the checkers to be enabled.

There is not group name and still optional arguments. And the command line does not show exclusive. I believe they are not designed to work like this.
msg264839 - (view) Author: Endre (Endre) Date: 2016-05-04 16:28

Thanks for taking a look at it.
I have only tried it with argparse-1.4.0.

"I don't think add_mutually_exclusive_group and add_argument_group are designed to work together."

Yes, it really works strange in this case. I guess the framework should raise an exception which states that this use case is not supported, e.g. at the "serverGroup = mutexGroup.add_argument_group('serverGroup')
" line.
msg265164 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2016-05-08 20:22
Argument Groups are not designed for nesting, and despite their names and subclassing, Mutually exclusive groups and Argument Groups are not meant to be used together (with one exception).

I agree that the error is obscure, but it occurs in a particularly fragile function in the formatter, '_format_actions_usage'.  That function needs a major rewrite (that's in another bug/issue).

Argument Groups serve only as a way of grouping help lines.  Mutually exclusive groups test arguments during parsing, and add some markings to the usage line.  So they have very different purposes.

I have seen questions on Stackoverflow where people try to use Argument Groups as a way of adding some sort of subgroup to the Mutually Exclusive one, one for example that implements a 'allow any of this group' logic.  There is a bug/issue asking for 'inclusive' nesting groups, but production patch of that sort is long ways off.

This usage line

    (--url URL --project Prj [--dump]) | (--mergeInput input.txt [--removeDisabled])

suggests that this what you are trying do - allow any of -u,-p,-d, but disallow one of these with -m or -r.  That logic is beyond the current group testing mechanism, and beyond the usage formatting code.  You'll have to do your own tests, and write a custom usage line.

Mutually exclusive groups can be nested in other mutual groups, but the effect isn't what you might hope.  It just forms a larger mutually exclusive group; there's no subgrouping.

It is possible to nest a mutually exclusive group in an Argument group; the effect is to give the mutually exclusive group a title.
Date User Action Args
2018-07-11 07:43:28serhiy.storchakasettype: crash -> behavior
2016-05-08 20:22:45paul.j3setnosy: + paul.j3
messages: + msg265164
2016-05-04 16:30:54xiang.zhangsetnosy: + bethard
2016-05-04 16:28:41Endresetmessages: + msg264839
2016-05-04 16:23:49xiang.zhangsetmessages: + msg264837
2016-05-04 16:15:10xiang.zhangsetmessages: + msg264835
2016-05-04 16:14:41berker.peksagsetnosy: + berker.peksag
messages: + msg264834
2016-05-04 16:01:22xiang.zhangsetnosy: + xiang.zhang
2016-05-04 15:14:47Endrecreate