Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

argparse add_mutually_exclusive_group should accept existing arguments to register conflicts #55193

Open
gotgenes mannequin opened this issue Jan 22, 2011 · 17 comments
Open
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@gotgenes
Copy link
Mannequin

gotgenes mannequin commented Jan 22, 2011

BPO 10984
Files
  • args_in_multiple_mutually_exclusive_groups.patch: Allow arguments to be passed into add_mutually_exclusive_group
  • multigroup_1.patch
  • multigroup_4.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2011-01-22.23:01:55.200>
    labels = ['type-feature', 'library']
    title = 'argparse add_mutually_exclusive_group should accept existing arguments to register conflicts'
    updated_at = <Date 2014-06-03.05:44:01.029>
    user = 'https://bugs.python.org/gotgenes'

    bugs.python.org fields:

    activity = <Date 2014-06-03.05:44:01.029>
    actor = 'paul.j3'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2011-01-22.23:01:55.200>
    creator = 'gotgenes'
    dependencies = []
    files = ['30818', '30921', '30940']
    hgrepos = []
    issue_num = 10984
    keywords = ['patch']
    message_count = 13.0
    messages = ['126862', '126879', '192444', '192765', '192844', '192954', '193071', '193184', '196771', '197960', '212458', '212482', '219660']
    nosy_count = 5.0
    nosy_names = ['bethard', 'gotgenes', 'micktwomey', 'paul.j3', 'jamadagni']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = None
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue10984'
    versions = ['Python 3.3']

    @gotgenes
    Copy link
    Mannequin Author

    gotgenes mannequin commented Jan 22, 2011

    argparse supports registering conflicting arguments, however, it does so in a way that an argument may belong to at most one group of conflicting arguments. The inspiration for this bug is Stack Overflow question #4770576.
    http://stackoverflow.com/questions/4770576/does-argparse-python-support-mutually-exclusive-groups-of-arguments

    The most straightforward use case argparse can not accommodate is the following: the user has three flags, '-a', '-b', and '-c'. The flag '-b' is incompatible with both '-a' and with '-c', however, '-a' and '-c' are compatible with each other.

    Current practice is to register a conflict by first defining a conflict group with parser.add_mutually_exclusive_group(), and then create new arguments within that group using group.add_argument(). Because the programmer is not allowed to create the argument prior to creating the group, an argument cannot be registered in two exclusive groups.

    I feel it would be much more useful to be given the option to create exclusive groups after the programmer has defined and created the options, as is the design for ConflictsOptionParser
    http://pypi.python.org/pypi/ConflictsOptionParser/

    @gotgenes gotgenes mannequin added type-bug An unexpected behavior, bug, or error stdlib Python modules in the Lib dir labels Jan 22, 2011
    @bethard
    Copy link
    Mannequin

    bethard mannequin commented Jan 23, 2011

    I'm definitely open to providing such functionality. I assume you're imagining something like:

    parser = argparse.ArgumentParser()
    a_action = parser.add_argument('-a')
    b_action = parser.add_argument('-b')
    c_action = parser.add_argument('-c')
    d_action = parser.add_argument('-d')
    parser.add_mutually_exclusive_group(a_action, c_action)
    parser.add_mutually_exclusive_group(a_action, d_action)
    ...

    If you can supply a patch, I'll take a look at it.

    @bethard bethard mannequin added type-feature A feature request or enhancement and removed type-bug An unexpected behavior, bug, or error labels Jan 23, 2011
    @micktwomey
    Copy link
    Mannequin

    micktwomey mannequin commented Jul 6, 2013

    I've picked up on this as part of the EP 2013 sprints.

    I've attached a patch which implements the behaviour described in the comments. It doesn't break any existing functionality but the help generated by argparse is definitely not quite right.

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Jul 9, 2013

    This approach of simply adding the existing actions to the group's _group_actions works fine, at least when it comes catching the error.

    It may be difficult to get a useful usage line. In usage, arguments appear in the order in which they were created, optionals first, positionals after. Group notation is added if the subset of the arguments appear in its list in the same order.

    In the patch .rst,

    usage: PROG [-h] (--foo | --bar) [--baz]
    

    the foo,bar group is marked correctly; the foo,baz group is not contiguous and is omited.

    In bethard's example neither group will be marked
    (class TestMutuallyExclusiveGroupWithExistingArguments)

    But the problem isn't just with adding existing arguments.

    class TestMutuallyExclusiveOptionalsMixed illustrates this with a case where group and parser arguments overlap.

    In class TestMutuallyExclusiveOptionalsAndPositionalsMixed, the mix of optionals and positionals makes group marking impossible.

    If the groups are in order, but overlap, usage can be a confusing mix

    Groups ab, bc, cd, produce:
    [-a A | [-b B | [-c C | -d D]

    But if created in a different order, the usage can be:
    [-a A | [-b B | -c C] -d D]

    So there are 2 issues

    • if groups are not continuous or overlap, what is a meaningful usage?
    • any choice is likely to require a major reworking of the formatting logic.

    Since confusing group markings are worse than none, a first step might be to flag a group added via this patch as 'do not mark'. Also add a note to the documentation that user may need to write their own grouping instructions (in usage, description or epilog).

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Jul 11, 2013

    One usage option is to make a subclass of HelpFormatter (the accepted way of customizing a formatter), and write a function that formats each group independently. For the example case, the resulting format might be:

    usage: PROG [-h] [-b] [-a | -c] [-a | -d]

    -h and -b are not part of any group. These are followed by the two groups. -a is repeated because it appears in both groups.

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Jul 12, 2013

    While playing with some examples, I found that exclusive group formatting has another failure case. If the usage line is long enough to wrap, optionals and positionals are formatted separately, with positionals appearing on a separate line(s). That means that if a group includes a positional, it will not be marked.

    So (shortening lines for convenience sake), instead of:

    usage: [-h] ... (-a | -b | x)
    

    we get

    usage: [-h] ... [-a] [-b]
           x
    

    This is true even if arguments are added to the group in the normal way.

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Jul 15, 2013

    This patch adds a MultiGroupHelpFormatter that formats groups even if they share actions or the actions are not in the same order as in the parse._actions list. It sorts the groups so positional actions, if any appear in the correct order.

    A long test case generates this help:

    usage: PROG [-h] [-a A | -c C] [-a A | -d D] [-a A | -b B] [-b B | -d D]
                [-d D | x] foo [-b B | y]
    
    positional arguments:
      x           x help
      foo         foo help        
      y           y help
    
    optional arguments:
      -h, --help  show this help message and exit
      -a A        a help
      -b B        b help
      -c C        c help
      -d D        d help
    

    In the 2nd usage line, the 2 groups, and action foo, are shown in the order in which x, foo, y were defined (and hence will be parsed), even though the groups were not defined in that order.

    The default formatter could not format these groups, generating '[-h] [-a A] [-b B] ... x foo y' instead.

    I have included the latest patch from http://bugs.python.org/issue11874. This splits the usage line generated by _format_actions_usage into parts that are groups or independent actions. The goal there is to correctly split long usage lines into multiple lines. Here it makes it easier to format groups and actions in new order.

    If existing actions are added to new group as in the original patch for this issue, that group gets a no_usage = True attribute. The default formatter then will not attempt to format this group. The MultiGroupHelpFormatter ignores this attribute.

    This patch needs better documentation. Test cases also need refinement, improving the names, and eliminating redundancies. Some of the new tests are copies of existing ones, but using the new formatter.

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Jul 16, 2013

    This patch produces the same usage as before, but I have rewritten _format_actions_usage() for both HelpFormatter and MultiGroupFormater.

    The original HelpFormatter._format_actions_usage() formats the actions, splices in group markings, cleans up the text, if needed, tries to break it down into parts. But this is fragile, as shown here and in issues 11874, 18349).

    Now _format_group_usage() and _format_just_actions_usage() format groups and actions directly, without the splice and divide steps. _format_actions_usage() for both classes call these to build a list of usage parts.

    This change also solves http://bugs.python.org/issue11874 and http://bugs.python.org/issue18349, since it does not have to break up a formatted text line (and in the process get confused by [] and ()).

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Sep 2, 2013

    A possible further tweak is, in take_action(), test for conflicts before adding the action to 'seen_non_default_actions'

                if argument_values is not action.default:
                    #seen_non_default_actions.add(action)
                    for conflict_action in action_conflicts.get(action, []):
                        if conflict_action in seen_non_default_actions:
                           ...
                    seen_non_default_actions.add(action)

    This does not cause problems with any existing tests, but makes it possible to add an action twice to a group. Why do that? To prevent an action from occurring more than once. For some actions like 'count' and 'append' repeated use is expected, but for others it isn't expected, and may sometimes be a nuisance (the last occurrence is the one that sticks).

    An example use would be:

        parser = argparse.ArgumentParser(prog="PROG",
            formatter_class=argparse.MultiGroupHelpFormatter)
        action = parser.add_argument('--arg', help='use this argument only once')
        group1 = parser.add_mutually_exclusive_group(action, action)
        args  = parser.parse_args()

    calling this with:

    python3 test_once.py --arg test --arg next
    

    would produce this error message:

    usage: PROG [-h] [--arg ARG | --arg ARG]
    PROG: error: argument --arg: not allowed with argument --arg
    

    The usage and error message aren't as clear as they might be if this feature was added 'from scratch'. But for a minor change like this, that may be an acceptable price.

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Sep 17, 2013

    Another situation in which this MultiGroupHelpFormatter helps is when one or more of the groups includes an optional positional.

    The regular formatter moves all the positionals to the end, after the optionals. This move could easily break up a mutually exclusive group, and make formatting it impossible. But the MultiGroupHelpFormatter gives the group order priority.

    Thus for example:

        p=argparse.ArgumentParser()
        # (formatter_class=argparse.MultiGroupHelpFormatter)
        g=p.add_mutually_exclusive_group()
        g.add_argument('-f')
        g.add_argument('foo',nargs='?')
        g=p.add_mutually_exclusive_group()
        g.add_argument('-b')
        g.add_argument('-c')
        g.add_argument('bar',nargs='*',default='X')
        print(p.format_usage())

    produces (positionals at end, no group markings)

    usage: PROG [-h] [-f F] [-b B] [-c C] [foo] [bar [bar ...]]
    

    But the MultiGroupHelpFormatter produces:

    usage: PROG [-h] [-f F | foo] [-b B | -c C | bar [bar ...]]
    

    In this last case, the positionals are listed with their respective groups, and the groups are ordered by the relative ordering of the positionals.

    @jamadagni
    Copy link
    Mannequin

    jamadagni mannequin commented Feb 28, 2014

    I also wish to see argparse allowing me to define a group of arguments that conflict with another argument or another group of arguments and FWIW I feel the help output should be like:

    prog [ --conflicter | [ --opt1 ] [ --opt2 ] ]

    where --conflicter conflicts with --opt1 and --opt2 but those two don't conflict with each other and all are optional.

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Feb 28, 2014

    In http://bugs.python.org/issue11588 (Add "necessarily inclusive" groups to argparse) I propose a generalization to these testing groups that would solve your 'conflicter' case as follows:

        usage = 'prog [ --conflicter | [ --opt1 ] [ --opt2 ] ]'
        parser = argparse.ArgumentParser(usage=usage)
        conflicter = parser.add_argument("--conflicter", action='store_true')
        opt1 = parser.add_argument("--opt1", action='store_true')
        opt2 = parser.add_argument("--opt2", action='store_true')
    
        @parser.crosstest
        def test(parser, seen_actions, *args):
            if conflicter in seen_actions:
                if 0<len(seen_actions.intersection([opt1, opt2])):
                    parser.error('--conflicter cannot be used with --opt1 or --opt2')

    Groups, as currently defined, cannot handle nesting, and as a consequence cannot handle complex logic. My proposal is to replace groups with user defined conflict tests that would be run near the end of 'parse_args'.

    This example shows, I think, that the proposal is powerful enough. I'm not sure about ease of use and logical transparency.

    Formatting the usage line is a different issue, though the MultiGroupHelpFormatter that I propose here is a step in the right direction. For now a user written 'usage' is the simplest solution.

    @paulj3
    Copy link
    Mannequin

    paulj3 mannequin commented Jun 3, 2014

    Another way to add an existing Action to a group is to modify the 'add_argument' method for the Group subclass. For example we could add this to the _MutuallyExclusiveGroup class:

        def add_argument(self, *args, **kwargs):
            # allow adding a prexisting Action
            if len(args) and isinstance(args[0], Action):
                action =  args[0]
                return self._group_actions.append(action)
            else:
                return super(_MutuallyExclusiveGroup, self).add_argument(*args, **kwargs)

    With this the 1st example might be written as:

        group1 = parser.add_mutually_exclusive_group()
        a_action = parser.add_argument('-a')
        c_action = parser.add_argument('-c')
        group2 = parser.add_mutually_exclusive_group()
        group2.add_argument(a_action)
        d_action = parser.add_argument('-d')

    This might be more intuitive to users.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @avalonv
    Copy link

    avalonv commented Dec 5, 2022

    I also wish to see argparse allowing me to define a group of arguments that conflict with another argument or another group of arguments and FWIW I feel the help output should be like:

    prog [ --conflicter | [ --opt1 ] [ --opt2 ] ]

    where --conflicter conflicts with --opt1 and --opt2 but those two don't conflict with each other and all are optional.

    I know this is a very old issue, but this, to me, is an important feature, and I think it would be very nice if this issue could be re-visited.

    The simplest use case could be wanting to break down your arguments into distinct groups visually, which is very useful when one has a large set of options as it aids readability, and still have mutually exclusive options. This doesn't seem to be possible without modifying argparse classes, which adds some overheard to users.

    I haven't read the code, but it appears that "group" and "mutually exclusive group" are themselves... mutually exclusive. When logically, there isn't a good reason for them to be? It's natural to want to group options together. That doesn't mean those options can't have mutually exclusive relationships within themselves.

    @avalonv
    Copy link

    avalonv commented Dec 5, 2022

    Currently, the way to have both groups and mutually exclusive groups without modifying the base class is to individually check and test each arg individually. If you have a dozen mutually exclusive args, that can easily add another 50 lines of code. It's not very practical :(

    @avalonv
    Copy link

    avalonv commented Dec 5, 2022

    @rhettinger tagging you as you seem to be the maintainer for this project. It appears @micktwomey wrote a patch which fixes this back in 2013. I imagine the code has changed a lot since this was first requested, but is there any chance of this being implemented?

    avalonv added a commit to avalonv/reCBZ that referenced this issue Dec 5, 2022
    breaking options into categories is incompatible with mutually exclusive options in argparse, so there's a hacky stopgap in the meantime.
    see python/cpython#55193
    @LeggoMahEggo
    Copy link

    I have just encountered this problem -- while I can write a workaround to functionally accomplish the desired behavior, I would prefer to do this via exclusive groups.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    Status: Features
    Development

    No branches or pull requests

    2 participants