This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: argparse: allow add_mutually_exclusive_group on add_argument_group
Type: enhancement Stage:
Components: Library (Lib) Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: calestyo, fmigneault, paul.j3, rhettinger
Priority: normal Keywords:

Created on 2021-02-19 02:28 by calestyo, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
test.py calestyo, 2021-02-19 12:53
test-no-parent.py calestyo, 2021-02-19 12:53
issue43259_utility.py paul.j3, 2021-02-22 05:30
Messages (7)
msg387278 - (view) Author: Christoph Anton Mitterer (calestyo) Date: 2021-02-19 02:28
Hey.

AFAIU, the sole purpose of ArgumentParser.add_argument_group() is for the grouping within the help output.

It would be nice, if one could create a mutually exclusive group (with ArgumentParser.add_mutually_exclusive_group) from/within such a "normal" group, so that the mutually exclusive arguments are listed within the group, but are still, mutually exclusive.

Right now when doing something like:
    parser = argparse.ArgumentParser()
    parser_group = parser.add_argument_group("INPUT OPTIONS")
    parser_group_mutually_exclusive = parser_group.add_mutually_exclusive_group(required=False)
    parser_group_mutually_exclusive.add_argument("--from-args")
    parser_group_mutually_exclusive.add_argument("--from-files")
    parser_group_mutually_exclusive.add_argument("--from-stdin")
    parser_group.add_argument("-0", help="null delimited pathnames")

it works, but the mutually exclusive options are note printed within the "INPUT OPTIONS", but rather at the normal "optional arguments:" section of the help.

The above example also kinda shows what this could be used for:
- one might have e.g. a group for input options, and amongst that the mutually exclusive "--from-*" which specify the source of the input.


Cheers,
Chris.
msg387283 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2021-02-19 05:54
The mutually exclusive arguments are displayed with in the argument group, at least in my testing.  From a copy-n-paste of your example:

In [8]: parser.print_help()
usage: ipython3 [-h]
                [--from-args FROM_ARGS | --from-files FROM_FILES | --from-stdin FROM_STDIN]
                [-0 0]

optional arguments:
  -h, --help            show this help message and exit

INPUT OPTIONS:
  --from-args FROM_ARGS
  --from-files FROM_FILES
  --from-stdin FROM_STDIN
  -0 0                  null delimited pathnames


I've had occasion to note that this is the only kind of group nesting that works (or makes sense).

In add_container_actions, there is a comment:

        # add container's mutually exclusive groups
        # NOTE: if add_mutually_exclusive_group ever gains title= and
        # description= then this code will need to be expanded as above

So the original developer envisioned giving a mutually exclusive group a formatting role, but with this nesting this isn't needed (that I can tell).  But I don't think this has been documented.
msg387316 - (view) Author: Christoph Anton Mitterer (calestyo) Date: 2021-02-19 12:53
Okay the problem seems to be that I didn't give you the exact details on what I do.

Actually, the group (which then contains the mutually exclusive group) is contained in a "shared" parent parser, which I then use in the subparsers.

If I leave out the parent parser, then it works as you say.


I'll attach exact code (with which I can reproduce it).
msg387333 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2021-02-19 17:31
The parents mechanism is not elaborate.  It copies groups and actions by reference.  The comments that I quoted actually come from that method that does this copying.  

From a quick glance at that code I see that it does not preserve the group nesting.  Mutually_exclusive groups are added directly the parser.

Parents is primarily a convenience tool, especially if used entirely with your own code.  It's most valuable when importing the parent, and you don't have direct access to the code that constructed it.  But it seems to be used most often as a way of creating a number of similar subparsers.  For that it can be easily replaced with your own utility function(s).  There's no virtue in trying to do everything with the tools that argparse provides.
msg387481 - (view) Author: Christoph Anton Mitterer (calestyo) Date: 2021-02-21 23:52
Well but if that's anyway one of its actual major use cases, wouldn't it make sense to properly support it?

Especially when one has a large set of identical options (which is then even more likely to also include mutually exclusive ones) such a feature seems to be pretty useful to prevent bloated code by copy&pasting large number of lines of identical argument parsing code.


argparse is really nice and powerful,... and enables people to make clean argparsing code, but it seems a few quite features which are quite often asked for miss, like e.g. #11354 or that one can arbitrarily group mutually exclusive options like:

<mutually exclusive>
<group>
--foo
</group>
<group>
--bar
--baz
</group>
<(mutually exclusive>


Anyway, feel free to close if you don't like supporting mutually exclusive groups with parents.

Cheers,
Chris.
msg387499 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2021-02-22 05:30
I've added a script that does what you want, but with a simple utility function instead of a parent (or lots of copy-n-paste).

====

I explored the code a bit, and have an idea that might correct the [parent] behavior.

In the method that copies a parent's groups and actions

     def _add_container_actions(self, container):

It might be enough to change

        for group in container._mutually_exclusive_groups:
            mutex_group = self.add_mutually_exclusive_group(
                required=group.required)

to

        for group in container._mutually_exclusive_groups:
            pgroup = group._container
            mutex_group = pgroup.add_mutually_exclusive_group(
                required=group.required)

The mutually group records where it was create in its '._container' attribute.  Usually that would a parser, but in your example would the 'inputs' action group.  

I haven't tested this idea.

====

In https://bugs.python.org/issue11588 (request for inclusive groups), I explored adding a Usage_Group class that could nest.  That project become too large, especially when considering help formatting.  And I did not give any thought to dealing with parents there.

====

Another issue involving parents (and the potential problems caused by copy-by-reference).

https://bugs.python.org/issue22401 
argparse: 'resolve' conflict handler damages the actions of the parent parser

====

Belatedly I look for other issues involving 'parent', and found these duplicates

https://bugs.python.org/issue25882
argparse help error: arguments created by add_mutually_exclusive_group() are shown outside their parent group created by add_argument_group()

https://bugs.python.org/issue16807
argparse group nesting lost on inheritance
msg395959 - (view) Author: Francis Charette Migneault (fmigneault) Date: 2021-06-16 20:49
I have found that the only thing argparse is actually missing is to forward the actions to the generated mutually exclusive group in _add_container_actions method.  


    class SubArgumentParserFixedMutExtGroups(argparse.ArgumentParser):    
        def _add_container_actions(self, container):
            groups = container._mutually_exclusive_groups
            container._mutually_exclusive_groups = []  # temporary override just so it is not processed
            super(SubArgumentParserFixedMutexGroups, self)._add_container_actions(container)
            # same as original loop, but with extra append of actions in created mutex
            for group in groups:
                mutex_group = self.add_mutually_exclusive_group(required=group.required)
                for action in group._group_actions:
                    mutex_group._group_actions.append(action)


When printing help, this resolves correctly. 
The actions can be found because they are already added when parsing the group of the parent parser that contains the mutex.
Snippet below. 

    # same as other comment examples
    parser = argparse.ArgumentParser()
    parser_group = parser.add_argument_group("INPUT OPTIONS")
    parser_group_mutually_exclusive = parser_group.add_mutually_exclusive_group(required=False)
    parser_group_mutually_exclusive.add_argument("--from-args")
    parser_group_mutually_exclusive.add_argument("--from-files")
    parser_group_mutually_exclusive.add_argument("--from-stdin")
    parser_group.add_argument("-0", help="null delimited pathnames")
    parser.print_help()

    usage: pydevconsole.py [-h]
                           [--from-args FROM_ARGS | --from-files FROM_FILES | --from-stdin FROM_STDIN]
                           [-0 0]
    optional arguments:
      -h, --help            show this help message and exit
    INPUT OPTIONS:
      --from-args FROM_ARGS
      --from-files FROM_FILES
      --from-stdin FROM_STDIN
      -0 0                  null delimited pathnames

    # now add the subparser with tweaked ArgumentParser
    parent = SubArgumentParserFixedMutexGroups()
    subpar = parent.add_subparsers()
    subpar.add_parser("test", add_help=False, parents=[parser])
    parent.parse_args(["test", "--help"])

    usage: pydevconsole.py test [-h]
                                [--from-args FROM_ARGS | --from-files FROM_FILES | --from-stdin FROM_STDIN]
                                [-0 0]
    optional arguments:
      -h, --help            show this help message and exit
    INPUT OPTIONS:
      --from-args FROM_ARGS
      --from-files FROM_FILES
      --from-stdin FROM_STDIN
      -0 0                  null delimited pathnames
History
Date User Action Args
2022-04-11 14:59:41adminsetgithub: 87425
2021-06-16 20:49:44fmigneaultsetnosy: + fmigneault
messages: + msg395959
2021-02-22 05:30:13paul.j3setfiles: + issue43259_utility.py

messages: + msg387499
2021-02-21 23:52:23calestyosetmessages: + msg387481
2021-02-19 17:31:04paul.j3setmessages: + msg387333
2021-02-19 12:53:40calestyosetfiles: + test-no-parent.py
2021-02-19 12:53:32calestyosetfiles: + test.py

messages: + msg387316
2021-02-19 05:54:42paul.j3setmessages: + msg387283
2021-02-19 04:42:35shihai1991setnosy: + rhettinger, paul.j3
2021-02-19 02:28:41calestyocreate