Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

argparse: allow add_mutually_exclusive_group on add_argument_group #87425

Open
calestyo mannequin opened this issue Feb 19, 2021 · 7 comments
Open

argparse: allow add_mutually_exclusive_group on add_argument_group #87425

calestyo mannequin opened this issue Feb 19, 2021 · 7 comments
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@calestyo
Copy link
Mannequin

calestyo mannequin commented Feb 19, 2021

BPO 43259
Nosy @rhettinger, @fmigneault
Files
  • test.py
  • test-no-parent.py
  • issue43259_utility.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2021-02-19.02:28:41.907>
    labels = ['type-feature', 'library']
    title = 'argparse: allow add_mutually_exclusive_group on add_argument_group'
    updated_at = <Date 2021-06-16.20:49:44.489>
    user = 'https://bugs.python.org/calestyo'

    bugs.python.org fields:

    activity = <Date 2021-06-16.20:49:44.489>
    actor = 'fmigneault'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2021-02-19.02:28:41.907>
    creator = 'calestyo'
    dependencies = []
    files = ['49821', '49822', '49827']
    hgrepos = []
    issue_num = 43259
    keywords = []
    message_count = 7.0
    messages = ['387278', '387283', '387316', '387333', '387481', '387499', '395959']
    nosy_count = 4.0
    nosy_names = ['rhettinger', 'paul.j3', 'calestyo', 'fmigneault']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = None
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue43259'
    versions = []

    @calestyo
    Copy link
    Mannequin Author

    calestyo mannequin commented Feb 19, 2021

    Hey.

    AFAIU, the sole purpose of ArgumentParser.add_argument_group() is for the grouping within the help output.

    It would be nice, if one could create a mutually exclusive group (with ArgumentParser.add_mutually_exclusive_group) from/within such a "normal" group, so that the mutually exclusive arguments are listed within the group, but are still, mutually exclusive.

    Right now when doing something like:
    parser = argparse.ArgumentParser()
    parser_group = parser.add_argument_group("INPUT OPTIONS")
    parser_group_mutually_exclusive = parser_group.add_mutually_exclusive_group(required=False)
    parser_group_mutually_exclusive.add_argument("--from-args")
    parser_group_mutually_exclusive.add_argument("--from-files")
    parser_group_mutually_exclusive.add_argument("--from-stdin")
    parser_group.add_argument("-0", help="null delimited pathnames")

    it works, but the mutually exclusive options are note printed within the "INPUT OPTIONS", but rather at the normal "optional arguments:" section of the help.

    The above example also kinda shows what this could be used for:

    • one might have e.g. a group for input options, and amongst that the mutually exclusive "--from-*" which specify the source of the input.

    Cheers,
    Chris.

    @calestyo calestyo mannequin added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Feb 19, 2021
    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Feb 19, 2021

    The mutually exclusive arguments are displayed with in the argument group, at least in my testing. From a copy-n-paste of your example:

    In [8]: parser.print_help()
    usage: ipython3 [-h]
    [--from-args FROM_ARGS | --from-files FROM_FILES | --from-stdin FROM_STDIN]
    [-0 0]

    optional arguments:
    -h, --help show this help message and exit

    INPUT OPTIONS:
    --from-args FROM_ARGS
    --from-files FROM_FILES
    --from-stdin FROM_STDIN
    -0 0 null delimited pathnames

    I've had occasion to note that this is the only kind of group nesting that works (or makes sense).

    In add_container_actions, there is a comment:

        # add container's mutually exclusive groups
        # NOTE: if add_mutually_exclusive_group ever gains title= and
        # description= then this code will need to be expanded as above
    

    So the original developer envisioned giving a mutually exclusive group a formatting role, but with this nesting this isn't needed (that I can tell). But I don't think this has been documented.

    @calestyo
    Copy link
    Mannequin Author

    calestyo mannequin commented Feb 19, 2021

    Okay the problem seems to be that I didn't give you the exact details on what I do.

    Actually, the group (which then contains the mutually exclusive group) is contained in a "shared" parent parser, which I then use in the subparsers.

    If I leave out the parent parser, then it works as you say.

    I'll attach exact code (with which I can reproduce it).

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Feb 19, 2021

    The parents mechanism is not elaborate. It copies groups and actions by reference. The comments that I quoted actually come from that method that does this copying.

    From a quick glance at that code I see that it does not preserve the group nesting. Mutually_exclusive groups are added directly the parser.

    Parents is primarily a convenience tool, especially if used entirely with your own code. It's most valuable when importing the parent, and you don't have direct access to the code that constructed it. But it seems to be used most often as a way of creating a number of similar subparsers. For that it can be easily replaced with your own utility function(s). There's no virtue in trying to do everything with the tools that argparse provides.

    @calestyo
    Copy link
    Mannequin Author

    calestyo mannequin commented Feb 21, 2021

    Well but if that's anyway one of its actual major use cases, wouldn't it make sense to properly support it?

    Especially when one has a large set of identical options (which is then even more likely to also include mutually exclusive ones) such a feature seems to be pretty useful to prevent bloated code by copy&pasting large number of lines of identical argument parsing code.

    argparse is really nice and powerful,... and enables people to make clean argparsing code, but it seems a few quite features which are quite often asked for miss, like e.g. bpo-11354 or that one can arbitrarily group mutually exclusive options like:

    <mutually exclusive>
    <group>
    --foo
    </group>
    <group>
    --bar
    --baz
    </group>
    <(mutually exclusive>

    Anyway, feel free to close if you don't like supporting mutually exclusive groups with parents.

    Cheers,
    Chris.

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Feb 22, 2021

    I've added a script that does what you want, but with a simple utility function instead of a parent (or lots of copy-n-paste).

    ====

    I explored the code a bit, and have an idea that might correct the [parent] behavior.

    In the method that copies a parent's groups and actions

         def _add_container_actions(self, container):

    It might be enough to change

            for group in container._mutually_exclusive_groups:
                mutex_group = self.add_mutually_exclusive_group(
                    required=group.required)

    to

            for group in container._mutually_exclusive_groups:
                pgroup = group._container
                mutex_group = pgroup.add_mutually_exclusive_group(
                    required=group.required)

    The mutually group records where it was create in its '._container' attribute. Usually that would a parser, but in your example would the 'inputs' action group.

    I haven't tested this idea.

    ====

    In https://bugs.python.org/issue11588 (request for inclusive groups), I explored adding a Usage_Group class that could nest. That project become too large, especially when considering help formatting. And I did not give any thought to dealing with parents there.

    ====

    Another issue involving parents (and the potential problems caused by copy-by-reference).

    https://bugs.python.org/issue22401
    argparse: 'resolve' conflict handler damages the actions of the parent parser

    ====

    Belatedly I look for other issues involving 'parent', and found these duplicates

    https://bugs.python.org/issue25882
    argparse help error: arguments created by add_mutually_exclusive_group() are shown outside their parent group created by add_argument_group()

    https://bugs.python.org/issue16807
    argparse group nesting lost on inheritance

    @fmigneault
    Copy link
    Mannequin

    fmigneault mannequin commented Jun 16, 2021

    I have found that the only thing argparse is actually missing is to forward the actions to the generated mutually exclusive group in _add_container_actions method.

        class SubArgumentParserFixedMutExtGroups(argparse.ArgumentParser):    
            def _add_container_actions(self, container):
                groups = container._mutually_exclusive_groups
                container._mutually_exclusive_groups = []  # temporary override just so it is not processed
                super(SubArgumentParserFixedMutexGroups, self)._add_container_actions(container)
                # same as original loop, but with extra append of actions in created mutex
                for group in groups:
                    mutex_group = self.add_mutually_exclusive_group(required=group.required)
                    for action in group._group_actions:
                        mutex_group._group_actions.append(action)

    When printing help, this resolves correctly.
    The actions can be found because they are already added when parsing the group of the parent parser that contains the mutex.
    Snippet below.

        # same as other comment examples
        parser = argparse.ArgumentParser()
        parser_group = parser.add_argument_group("INPUT OPTIONS")
        parser_group_mutually_exclusive = parser_group.add_mutually_exclusive_group(required=False)
        parser_group_mutually_exclusive.add_argument("--from-args")
        parser_group_mutually_exclusive.add_argument("--from-files")
        parser_group_mutually_exclusive.add_argument("--from-stdin")
        parser_group.add_argument("-0", help="null delimited pathnames")
        parser.print_help()
    usage: pydevconsole.py [-h]
                           [--from-args FROM_ARGS | --from-files FROM_FILES | --from-stdin FROM_STDIN]
                           [-0 0]
    optional arguments:
      -h, --help            show this help message and exit
    INPUT OPTIONS:
      --from-args FROM_ARGS
      --from-files FROM_FILES
      --from-stdin FROM_STDIN
      -0 0                  null delimited pathnames
    
        # now add the subparser with tweaked ArgumentParser
        parent = SubArgumentParserFixedMutexGroups()
        subpar = parent.add_subparsers()
        subpar.add_parser("test", add_help=False, parents=[parser])
        parent.parse_args(["test", "--help"])
    usage: pydevconsole.py test [-h]
                                [--from-args FROM_ARGS | --from-files FROM_FILES | --from-stdin FROM_STDIN]
                                [-0 0]
    optional arguments:
      -h, --help            show this help message and exit
    INPUT OPTIONS:
      --from-args FROM_ARGS
      --from-files FROM_FILES
      --from-stdin FROM_STDIN
      -0 0                  null delimited pathnames
    

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    Status: Features
    Development

    No branches or pull requests

    0 participants