Argument Groups are not designed for nesting, and despite their names and subclassing, Mutually exclusive groups and Argument Groups are not meant to be used together (with one exception).

I agree that the error is obscure, but it occurs in a particularly fragile function in the formatter, '_format_actions_usage'.  That function needs a major rewrite (that's in another bug/issue).

Argument Groups serve only as a way of grouping help lines.  Mutually exclusive groups test arguments during parsing, and add some markings to the usage line.  So they have very different purposes.

I have seen questions on Stackoverflow where people try to use Argument Groups as a way of adding some sort of subgroup to the Mutually Exclusive one, one for example that implements a 'allow any of this group' logic.  There is a bug/issue asking for 'inclusive' nesting groups, but production patch of that sort is long ways off.  http://bugs.python.org/issue11588

This usage line

    (--url URL --project Prj [--dump]) | (--mergeInput input.txt [--removeDisabled])

suggests that this what you are trying do - allow any of -u,-p,-d, but disallow one of these with -m or -r.  That logic is beyond the current group testing mechanism, and beyond the usage formatting code.  You'll have to do your own tests, and write a custom usage line.

Mutually exclusive groups can be nested in other mutual groups, but the effect isn't what you might hope.  It just forms a larger mutually exclusive group; there's no subgrouping.

It is possible to nest a mutually exclusive group in an Argument group; the effect is to give the mutually exclusive group a title.