Title: argparse add_mutually_exclusive_group should accept existing arguments to register conflicts
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.3
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bethard, gotgenes, jamadagni, micktwomey, paul.j3
Priority: normal Keywords: patch

Created on 2011-01-22 23:01 by gotgenes, last changed 2014-06-03 05:44 by paul.j3.

File name Uploaded Description Edit
args_in_multiple_mutually_exclusive_groups.patch micktwomey, 2013-07-06 14:19 Allow arguments to be passed into add_mutually_exclusive_group review
multigroup_1.patch paul.j3, 2013-07-15 03:51 review
multigroup_4.patch paul.j3, 2013-07-16 18:01 review
Messages (13)
msg126862 - (view) Author: Chris Lasher (gotgenes) Date: 2011-01-22 23:01
argparse supports registering conflicting arguments, however, it does so in a way that an argument may belong to at most one group of conflicting arguments. The inspiration for this bug is Stack Overflow question #4770576.

The most straightforward use case argparse can not accommodate is the following: the user has three flags, '-a', '-b', and '-c'. The flag '-b' is incompatible with both '-a' and with '-c', however, '-a' and '-c' are compatible with each other.

Current practice is to register a conflict by first defining a conflict group with parser.add_mutually_exclusive_group(), and then create new arguments within that group using group.add_argument(). Because the programmer is not allowed to create the argument prior to creating the group, an argument cannot be registered in two exclusive groups.

I feel it would be much more useful to be given the option to create exclusive groups after the programmer has defined and created the options, as is the design for ConflictsOptionParser
msg126879 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2011-01-23 09:44
I'm definitely open to providing such functionality. I assume you're imagining something like:

parser = argparse.ArgumentParser()
a_action = parser.add_argument('-a')
b_action = parser.add_argument('-b')
c_action = parser.add_argument('-c')
d_action = parser.add_argument('-d')
parser.add_mutually_exclusive_group(a_action, c_action)
parser.add_mutually_exclusive_group(a_action, d_action)

If you can supply a patch, I'll take a look at it.
msg192444 - (view) Author: Michael Twomey (micktwomey) * (Python committer) Date: 2013-07-06 14:19
I've picked up on this as part of the EP 2013 sprints.

I've attached a patch which implements the behaviour described in the comments. It doesn't break any existing functionality but the help generated by argparse is definitely not quite right.
msg192765 - (view) Author: paul j3 (paul.j3) * Date: 2013-07-09 16:39
This approach of simply adding the existing actions to the group's _group_actions works fine, at least when it comes catching the error.

It may be difficult to get a useful usage line.  In usage, arguments appear in the order in which they were created, optionals first, positionals after.  Group notation is added if the subset of the arguments appear in its list in the same order.

In the patch .rst, 

    usage: PROG [-h] (--foo | --bar) [--baz]

the foo,bar group is marked correctly; the foo,baz group is not contiguous and is omited.

In bethard's example neither group will be marked
(class TestMutuallyExclusiveGroupWithExistingArguments)

But the problem isn't just with adding existing arguments.

class TestMutuallyExclusiveOptionalsMixed illustrates this with a case where group and parser arguments overlap.

In class TestMutuallyExclusiveOptionalsAndPositionalsMixed, the mix of optionals and positionals makes group marking impossible.

If the groups are in order, but overlap, usage can be a confusing mix

Groups ab, bc, cd, produce: 
    [-a A | [-b B | [-c C | -d D]

But if created in a different order, the usage can be: 
    [-a A | [-b B | -c C] -d D]

So there are 2 issues
   - if groups are not continuous or overlap, what is a meaningful usage?
   - any choice is likely to require a major reworking of the formatting logic.

Since confusing group markings are worse than none, a first step might be to flag a group added via this patch as 'do not mark'.  Also add a note to the documentation that user may need to write their own grouping  instructions (in usage, description or epilog).
msg192844 - (view) Author: paul j3 (paul.j3) * Date: 2013-07-11 06:33
One usage option is to make a subclass of HelpFormatter (the accepted way of customizing a formatter), and write a function that formats each group independently.  For the example case, the resulting format might be:

   usage: PROG [-h] [-b] [-a | -c] [-a | -d]

-h and -b are not part of any group.  These are followed by the two groups.  -a is repeated because it appears in both groups.
msg192954 - (view) Author: paul j3 (paul.j3) * Date: 2013-07-12 18:18
While playing with some examples, I found that exclusive group formatting has another failure case.  If the usage line is long enough to wrap, optionals and positionals are formatted separately, with positionals appearing on a separate line(s).  That means that if a group includes a positional, it will not be marked.

So (shortening lines for convenience sake), instead of:

    usage: [-h] ... (-a | -b | x)

we get

    usage: [-h] ... [-a] [-b]

This is true even if arguments are added to the group in the normal way.
msg193071 - (view) Author: paul j3 (paul.j3) * Date: 2013-07-15 03:51
This patch adds a MultiGroupHelpFormatter that formats groups even if they share actions or the actions are not in the same order as in the parse._actions list.  It sorts the groups so positional actions, if any appear in the correct order.

A long test case generates this help:

    usage: PROG [-h] [-a A | -c C] [-a A | -d D] [-a A | -b B] [-b B | -d D]
                [-d D | x] foo [-b B | y]

    positional arguments:
      x           x help
      foo         foo help        
      y           y help

    optional arguments:
      -h, --help  show this help message and exit
      -a A        a help
      -b B        b help
      -c C        c help
      -d D        d help

In the 2nd usage line, the 2 groups, and action foo, are shown in the order in which x, foo, y were defined (and hence will be parsed), even though the groups were not defined in that order.

The default formatter could not format these groups, generating '[-h] [-a A] [-b B] ... x foo y' instead.

I have included the latest patch from  This splits the usage line generated by _format_actions_usage into parts that are groups or independent actions.  The goal there is to correctly split long usage lines into multiple lines.  Here it makes it easier to format groups and actions in new order.

If existing actions are added to new group as in the original patch for this issue, that group gets a no_usage = True attribute.  The default formatter then will not attempt to format this group. The MultiGroupHelpFormatter ignores this attribute.

This patch needs better documentation. Test cases also need refinement, improving the names, and eliminating redundancies.  Some of the new tests are copies of existing ones, but using the new formatter.
msg193184 - (view) Author: paul j3 (paul.j3) * Date: 2013-07-16 18:01
This patch produces the same usage as before, but I have rewritten _format_actions_usage() for both HelpFormatter and MultiGroupFormater.

The original HelpFormatter._format_actions_usage() formats the actions, splices in group markings, cleans up the text, if needed, tries to break it down into parts.  But this is fragile, as shown here and in issues 11874, 18349).

Now _format_group_usage() and _format_just_actions_usage() format groups and actions directly, without the splice and divide steps.  _format_actions_usage() for both classes call these to build a list of usage parts.

This change also solves and, since it does not have to break up a formatted text line (and in the process get confused by [] and ()).
msg196771 - (view) Author: paul j3 (paul.j3) * Date: 2013-09-02 04:51
A possible further tweak is, in take_action(), test for conflicts before adding the action to 'seen_non_default_actions'

            if argument_values is not action.default:
                for conflict_action in action_conflicts.get(action, []):
                    if conflict_action in seen_non_default_actions:

This does not cause problems with any existing tests, but makes it possible to add an action twice to a group.  Why do that?  To prevent an action from occurring more than once.  For some actions like 'count' and 'append' repeated use is expected, but for others it isn't expected, and may sometimes be a nuisance (the last occurrence is the one that sticks). 

An example use would be:

    parser = argparse.ArgumentParser(prog="PROG",
    action = parser.add_argument('--arg', help='use this argument only once')
    group1 = parser.add_mutually_exclusive_group(action, action)
    args  = parser.parse_args()

calling this with:

    python3 --arg test --arg next

would produce this error message:

    usage: PROG [-h] [--arg ARG | --arg ARG]
    PROG: error: argument --arg: not allowed with argument --arg

The usage and error message aren't as clear as they might be if this feature was added 'from scratch'.  But for a minor change like this, that may be an acceptable price.
msg197960 - (view) Author: paul j3 (paul.j3) * Date: 2013-09-17 05:05
Another situation in which this MultiGroupHelpFormatter helps is when one or more of the groups includes an optional positional.  

The regular formatter moves all the positionals to the end, after the optionals.  This move could easily break up a mutually exclusive group, and make formatting it impossible.  But the MultiGroupHelpFormatter gives the group order priority.

Thus for example:

    # (formatter_class=argparse.MultiGroupHelpFormatter)

produces (positionals at end, no group markings)

    usage: PROG [-h] [-f F] [-b B] [-c C] [foo] [bar [bar ...]]

But the MultiGroupHelpFormatter produces:

    usage: PROG [-h] [-f F | foo] [-b B | -c C | bar [bar ...]]

In this last case, the positionals are listed with their respective groups, and the groups are ordered by the relative ordering of the positionals.
msg212458 - (view) Author: Shriramana Sharma (jamadagni) Date: 2014-02-28 16:21
I also wish to see argparse allowing me to define a group of arguments that conflict with another argument or another group of arguments and FWIW I feel the help output should be like:

prog [ --conflicter | [ --opt1 ] [ --opt2 ] ]

where --conflicter conflicts with --opt1 and --opt2 but those two don't conflict with each other and all are optional.
msg212482 - (view) Author: paul j3 (paul.j3) * Date: 2014-02-28 23:04
In (Add "necessarily inclusive" groups to argparse) I propose a generalization to these testing groups that would solve your 'conflicter' case as follows:

    usage = 'prog [ --conflicter | [ --opt1 ] [ --opt2 ] ]'
    parser = argparse.ArgumentParser(usage=usage)
    conflicter = parser.add_argument("--conflicter", action='store_true')
    opt1 = parser.add_argument("--opt1", action='store_true')
    opt2 = parser.add_argument("--opt2", action='store_true')

    def test(parser, seen_actions, *args):
        if conflicter in seen_actions:
            if 0<len(seen_actions.intersection([opt1, opt2])):
                parser.error('--conflicter cannot be used with --opt1 or --opt2')

Groups, as currently defined, cannot handle nesting, and as a consequence cannot handle complex logic.  My proposal is to replace groups with user defined conflict tests that would be run near the end of 'parse_args'.  

This example shows, I think, that the proposal is powerful enough.  I'm not sure about ease of use and logical transparency.

Formatting the usage line is a different issue, though the MultiGroupHelpFormatter that I propose here is a step in the right direction.  For now a user written 'usage' is the simplest solution.
msg219660 - (view) Author: paul j3 (paul.j3) * Date: 2014-06-03 05:44
Another way to add an existing Action to a group is to modify the 'add_argument' method for the Group subclass.  For example we could add this to the _MutuallyExclusiveGroup class:

    def add_argument(self, *args, **kwargs):
        # allow adding a prexisting Action
        if len(args) and isinstance(args[0], Action):
            action =  args[0]
            return self._group_actions.append(action)
            return super(_MutuallyExclusiveGroup, self).add_argument(*args, **kwargs)

With this the 1st example might be written as:

    group1 = parser.add_mutually_exclusive_group()
    a_action = parser.add_argument('-a')
    c_action = parser.add_argument('-c')
    group2 = parser.add_mutually_exclusive_group()
    d_action = parser.add_argument('-d')

This might be more intuitive to users.
Date User Action Args
2014-06-03 05:44:01paul.j3setmessages: + msg219660
2014-02-28 23:04:45paul.j3setmessages: + msg212482
2014-02-28 16:21:00jamadagnisetmessages: + msg212458
2014-02-28 15:59:43jamadagnisetnosy: + jamadagni
2013-09-17 05:05:58paul.j3setmessages: + msg197960
2013-09-02 04:51:51paul.j3setmessages: + msg196771
2013-07-16 18:02:03paul.j3setfiles: + multigroup_4.patch

messages: + msg193184
2013-07-15 03:51:42paul.j3setfiles: + multigroup_1.patch

messages: + msg193071
2013-07-12 18:18:24paul.j3setmessages: + msg192954
2013-07-11 06:33:09paul.j3setmessages: + msg192844
2013-07-09 16:39:53paul.j3setnosy: + paul.j3
messages: + msg192765
2013-07-06 14:19:15micktwomeysetfiles: + args_in_multiple_mutually_exclusive_groups.patch

nosy: + micktwomey
messages: + msg192444

keywords: + patch
2011-01-23 09:44:36bethardsetnosy: bethard, gotgenes
type: behavior -> enhancement
messages: + msg126879
versions: - Python 3.1, Python 2.7, Python 3.2
2011-01-23 00:38:50r.david.murraysetnosy: + bethard
2011-01-22 23:01:55gotgenescreate