classification
Title: argparse AssertionError with add_mutually_exclusive_group and help=SUPPRESS
Type: behavior Stage: patch review
Components: Versions: Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: argparse: mutually exclusive groups full of help-suppressed args can cause AssertionErrors
View: 17890
Assigned To: Nosy List: bethard, mZarjk, martin.panter, paul.j3
Priority: normal Keywords: patch

Created on 2014-09-08 11:46 by mZarjk, last changed 2016-01-29 03:30 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
bug.py mZarjk, 2014-09-08 11:46 Minimal program to reproduce bug.
bugfix.patch mZarjk, 2014-09-09 03:22 Patch for the bug review
Messages (5)
msg226572 - (view) Author: (mZarjk) Date: 2014-09-08 11:46
Executing the attached script causes an AssertionError.

Traceback (most recent call last):
  File "bug.py", line 18, in <module>
    parser.format_usage()
  File "/usr/lib/python3.4/argparse.py", line 2318, in format_usage
    return formatter.format_help()
  File "/usr/lib/python3.4/argparse.py", line 287, in format_help
    help = self._root_section.format_help()
  File "/usr/lib/python3.4/argparse.py", line 217, in format_help
    func(*args)
  File "/usr/lib/python3.4/argparse.py", line 338, in _format_usage
    assert ' '.join(opt_parts) == opt_usage
AssertionError

The script was tested in a clean Python installation.

If any of the arguments are removed, there is no AssertionError exception.
If "help=SUPPRESS" is removed, there is no AssertionError exception.

This bug appears to have existed since Python 3.2, the first version that included argparse.
msg226618 - (view) Author: (mZarjk) Date: 2014-09-09 03:22
This assert statement is only reached when the usage line is long enough that it needs to be wrapped. Which is why the assertion does not happen when an argument is removed.

opt_usage is being compared to this string:
"[-h] [--arg2 ARG2] [--arg3 ARG3] [--arg4 ARG4] [--arg5 ARG5] [--arg6 ARG6]"

opt_usage in this case has the value:
"[-h]  [--arg2 ARG2] [--arg3 ARG3] [--arg4 ARG4] [--arg5 ARG5] [--arg6 ARG6]"

There is an extra space after "[-h]". The value of opt_usage comes from _format_actions_usage. And that is where the bug is.

Just before _format_actions_usage returns, the usage string is:
"[-h] [ ] [--arg2 ARG2] [--arg3 ARG3] [--arg4 ARG4] [--arg5 ARG5] [--arg6 ARG6]"

The method uses this regexp:
"[ *]"

To remove the extra brackets. But that leaves the extra space after the brackets.

The attached patch changes the regexp to:
"[ *] ?"

So that the extra space is also removed.

The patch also adds a testcase based on the script that reproduces the
msg226624 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-09-09 06:10
This assertion has triggered several bug issues, either due to an empty group like this, or certain characters in the metavars.

http://bugs.python.org/issue17890
'argparse: mutually exclusive groups full of suppressed args can cause AssertionErrors'

http://bugs.python.org/issue11874 
'argparse assertion failure with brackets in metavars'

http://bugs.python.org/issue16360 
'argparse: comma in metavar causes assertion failure when formatting long usage message'

I think this is a duplicate of 17890, though I'll have to look more closely at your patch to see what it adds.

For 11874 I proposed a major rewrite of the usage formatting.  There I format each group and/or action separately, and keep the pieces in a list.  That way it's easier to filter out empty pieces, and avoids the complex split that gives headaches when the usage is long enough to wrap.
msg226627 - (view) Author: (mZarjk) Date: 2014-09-09 07:48
I've confirmed that this is a duplicate of issue 17890.

In case anyone needs it, a workaround is to move the mutually exclusive group to the end of the arguments.
msg259188 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-01-29 03:30
I left one review comment, but it seems there is more discussion at Issue 17890, which also originally proposed a very similar fix.
History
Date User Action Args
2016-01-29 03:30:00martin.pantersetstatus: open -> closed

superseder: argparse: mutually exclusive groups full of help-suppressed args can cause AssertionErrors

nosy: + martin.panter
messages: + msg259188
resolution: duplicate
stage: patch review
2014-09-09 07:48:29mZarjksetmessages: + msg226627
2014-09-09 06:10:56paul.j3setnosy: + paul.j3
messages: + msg226624
2014-09-09 03:22:11mZarjksetfiles: + bugfix.patch
keywords: + patch
messages: + msg226618
2014-09-08 11:46:27mZarjkcreate