Issue42297
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020-11-09 17:25 by TurboTurtle, last changed 2022-04-11 14:59 by admin.
Messages (7) | |||
---|---|---|---|
msg380598 - (view) | Author: Jake Hunsaker (TurboTurtle) | Date: 2020-11-09 17:25 | |
In the sos project, we build a custom `usage` string for our argparser parser, and have noticed that doing so causes error messages from argparse to be badly formatted. For example if a bad option value is given, the error message is mangled into the last line of our usage string: ``` # python3 bin/sos report --all-logs=on usage: sos report [options] sos <component> [options] [..snip...] collect, collector Collect an sos report from multiple nodes simultaneously report: error: argument --all-logs: ignored explicit argument 'on' ``` This is especially strange since we build the usage string with a trailing newline character: ``` for com in self._components: aliases = self._components[com][1] aliases.insert(0, com) _com = ', '.join(aliases) desc = self._components[com][0].desc _com_string += ( "\t{com:<30}{desc}\n".format(com=_com, desc=desc) ) usage_string = ("%(prog)s <component> [options]\n\n" "Available components:\n") usage_string = usage_string + _com_string epilog = ("See `sos <component> --help` for more information") self.parser = ArgumentParser(usage=usage_string, epilog=epilog) ``` So it appears the trailing newlines are being stripped (in our case, unintentionally?). As expected, removing the trailing newline when passing `usage_string` to our parse does not change this behavior. However, if we don't set the usage string at all when instantiating our parser, the error message is properly formatted beginning on a new line. Slightly interesting is that without the usage_string being passed, the error message is prefixed with "sos: report:" as expected for %(prog)s expansion, but when the error message is mangled `%(prog)s` is left out as well. A little more context is available here: https://github.com/sosreport/sos/issues/2285 |
|||
msg380611 - (view) | Author: paul j3 (paul.j3) * ![]() |
Date: 2020-11-09 19:45 | |
Provide a minimal reproducible example. I can't reproduce that run on error message. Also test with arguments like '--all-logs on', which issues an 'unrecognizeable argument' error (with a different error reporting path). Stripping excess newlines is normal, both in the full help and error. That's done at the end of help formatting. |
|||
msg380612 - (view) | Author: Jake Hunsaker (TurboTurtle) | Date: 2020-11-09 19:56 | |
I'll try and get a simple reproducer made shortly, however as a quick note I've found that using '--all-logs on' results in a properly formatted error message. |
|||
msg380615 - (view) | Author: Jake Hunsaker (TurboTurtle) | Date: 2020-11-09 20:29 | |
Ah, ok - so I neglected to mention we're using subparsers which appears to be relevant here. My apologies. Here's a minimal reproducer that shows the behavior when using './arg_test.py foo --bar=on' ``` #! /bin/python3 import argparse usage_string = 'test usage string ending in newlines\n\n' sub_cmd_usage = '' for i in range(0, 3): sub_cmd_usage += '\tfoo --bar\n' usage_string += sub_cmd_usage parser = argparse.ArgumentParser(usage=usage_string) subparser = parser.add_subparsers(dest='subcmd', metavar='subcmd') subcmd_parser = subparser.add_parser('foo') subcmd_parser.add_argument('--bar', action="store_true", default=False) if __name__ == '__main__': args = parser.parse_args() ``` |
|||
msg380616 - (view) | Author: paul j3 (paul.j3) * ![]() |
Date: 2020-11-09 21:26 | |
It's the subparser that's producing this error, specifically its 'prog' attribute. If I use a custom usage with a simple parser: 1129:~/mypy$ python3 issue42297.py --foo=on usage: issue42297.py one two three issue42297.py: error: argument --foo: ignored explicit argument 'on' Notice that the error line includes the 'prog'. With subparsers, the main usage is included in the subcommand prog: print(subcmd_parser.prog) produces: test usage string ending in newlines foo --bar foo --bar foo --bar foo That's the usage plus the subcommand name, 'foo'. Generating the explicit error in the subcommand: 1244:~/mypy$ python3 issue42297.py foo --bar=on test usage string ending in newlines foo --bar foo --bar foo --bar foo: error: argument --bar: ignored explicit argument 'on' 'issue42297.py: ' has been replaced by the usage+'foo', and no newline. We don't see this in the 'unrecognized' case because that error issued by the main parser. issue42297.py: error: unrecognized arguments: on If I explicitly set the prog of the subcommand: subcmd_parser = subparser.add_parser('foo', prog='myscript foo') The error becomes: 1256:~/mypy$ python3 issue42297.py foo --bar=on usage: myscript foo [-h] [--bar] myscript foo: error: argument --bar: ignored explicit argument 'on' I can also add 'usage=usage_string' to the add_parser. For the most part add_parser takes the same parameters as ArgumentParser. Alternatively we can specify prog in subparser = parser.add_subparsers(dest='subcmd', metavar='subcmd', prog='myscript') resulting in: myscript foo: error: argument --bar: ignored explicit argument 'on' I recently explored how 'prog' is set with subparsers in https://bugs.python.org/issue41980 I don't think anything needs to be corrected in argparse. There are enough options for setting prog and usage in subcommands to get around this issue. In the worse case, you might want to create an alternative _SubParsersAction Action subclass that defines the prog/usage differently. |
|||
msg380619 - (view) | Author: Jake Hunsaker (TurboTurtle) | Date: 2020-11-09 22:03 | |
Ok, yeah there seem to be several paths to avoid this behavior then. We should be fine exploring those options. Thanks for the pointer! |
|||
msg380701 - (view) | Author: paul j3 (paul.j3) * ![]() |
Date: 2020-11-10 19:44 | |
We could look into using a different more compact 'prog' for these error messages. Currently the subparser 'prog' uses the main prog plus positionals plus the subparser name. The goal is to construct a subparser usage that reflects required input. This is fine for the usage line, but could be too verbose for some errors. In an error, the 'prog' just helps identify which argument has problems, and doesn't need the extra usage information. Most of the time that isn't an issue, since we don't use positional much in the main parser (and when used can't have variable nargs). But I don't have immediate ideas as to what can be conveniently (and safely) changed. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:37 | admin | set | github: 86463 |
2021-08-07 02:02:50 | bobblanchett | set | nosy:
+ bobblanchett |
2020-11-10 19:44:18 | paul.j3 | set | messages: + msg380701 |
2020-11-09 22:03:22 | TurboTurtle | set | messages: + msg380619 |
2020-11-09 21:26:16 | paul.j3 | set | messages: + msg380616 |
2020-11-09 20:29:35 | TurboTurtle | set | messages: + msg380615 |
2020-11-09 19:56:41 | TurboTurtle | set | messages: + msg380612 |
2020-11-09 19:45:37 | paul.j3 | set | messages: + msg380611 |
2020-11-09 17:29:05 | xtreak | set | nosy:
+ rhettinger, paul.j3 |
2020-11-09 17:25:59 | TurboTurtle | create |