Issue44748
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2021-07-27 09:26 by Thermi, last changed 2022-04-11 14:59 by admin.
Messages (14) | |||
---|---|---|---|
msg398288 - (view) | Author: (Thermi) | Date: 2021-07-27 09:26 | |
It'd be great if as part of the namespace returned by argparse.ArgumentParser.parse_args(), there was a bool indicating if a specific argument was encountered. That could then be used to implement the following behaviour: With a config file loaded as part of the program, overwrite the values loaded from the config file if the argument was encountered in the argument vector. That's necessary to implement overwriting of settings that were previously set by a different mechanism, e.g. read from a config file. After all the following order of significance should be used: program defaults < config file < argument vector |
|||
msg398351 - (view) | Author: 🖤Black Joker🖤 (joker) | Date: 2021-07-28 07:14 | |
I would like to use argparse to parse boolean command-line arguments written as "--foo True" or "--foo False". For example: my_program --my_boolean_flag False However, the following test code does not do what I would like: import argparse parser = argparse.ArgumentParser(description="My parser") parser.add_argument("--my_bool", type=bool) cmd_line = ["--my_bool", "False"] parsed_args = parser.parse(cmd_line) |
|||
msg398358 - (view) | Author: (Thermi) | Date: 2021-07-28 08:55 | |
joker, that is a different issue from the one described here. Please open your own. |
|||
msg398416 - (view) | Author: paul j3 (paul.j3) * | Date: 2021-07-28 19:47 | |
I've explored something similar in https://bugs.python.org/issue11588 Add "necessarily inclusive" groups to argparse There is a local variable in parser._parse_known_args seen_non_default_actions that's a set of the actions that have been seen. It is used for testing for required actions, and for mutually_exclusive groups. But making it available to users without altering user API code is awkward. My latest idea was to add it as an attribute to the parser, or (conditionally) as attribute of the namespace https://bugs.python.org/issue11588#msg265734 I've also thought about tweaking the interface between parser._parse_known_args parser.parse_known_args to do of more of the error checking in the caller, and give the user more opportunity to do their checks. This variable would be part of _parse_known_args output. Usually though when testing like this comes up on SO, I suggest leaving the defaults as None, and then just using a if args.foobar is None: # not seen Defaults are written to the namespace at the start of parsing, and seen arguments overwrite those values (with an added type 'eval' step of remaining defaults at the end). Keep in mind, though, that the use of subparsers could complicate any of these tweaks. In reading my posts on https://bugs.python.org/issue26394, I remembered the IPython uses argparse (subclassed) with config. I believe it uses config inputs (default and user) to define the arguments for the parser. So unless someone comes up with a really clever idea, this is bigger request than it first impressions suggest. |
|||
msg398417 - (view) | Author: paul j3 (paul.j3) * | Date: 2021-07-28 19:51 | |
Joker 'type=bool' has been discussed in other issues. 'bool' is an existing python function. Only 'bool("")' returns False. Write your own 'type' function if you want to test for specific strings. It's too language-specific to add as a general purpose function. |
|||
msg398456 - (view) | Author: paul j3 (paul.j3) * | Date: 2021-07-29 04:33 | |
More on the refactoring of error handling in _parse_known_args https://bugs.python.org/issue29670#msg288990 This is in a issue wanting better handling of the pre-populated "required" arguments, https://bugs.python.org/issue29670 argparse: does not respect required args pre-populated into namespace |
|||
msg398608 - (view) | Author: Terry J. Reedy (terry.reedy) * | Date: 2021-07-30 20:41 | |
Joker, please don't mess with headers. Enhancements only appear in future versions; argparse is a library module, not a test module. |
|||
msg400957 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2021-09-02 20:55 | |
> With a config file loaded as part of the program, > overwrite the values loaded from the config file > if the argument was encountered in the argument vector. It seems to me that default values can already be used for this purpose: from argparse import ArgumentParser config = {'w': 5, 'x': 10, 'y': False, 'z': True} missing = object() p = ArgumentParser() p.add_argument('-x', type=int, default=missing) p.add_argument('-y', action='store_true', default=missing) ns = p.parse_args() # update config for specified values for parameter, value in vars(ns).items(): if value is not missing: config[parameter] = value print(config) |
|||
msg400958 - (view) | Author: (Thermi) | Date: 2021-09-02 21:07 | |
Raymond, then you can't show the defaults in the help message. |
|||
msg400968 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2021-09-02 23:28 | |
> then you can't show the defaults in the help message. 1) The --help option doesn't normally show defaults. 2) Why would you show defaults in help, if you're going to ignore them in favor the values in config whenever they aren't specified. If ignored or overridden, they aren't actually default values. 3) Why not dynamically configure the argparse default values with data from config? config = {'w': 5, 'x': 10, 'y': False, 'z': True} p = ArgumentParser() p.add_argument('-x', type=int, default=config['x']) p.add_argument('-y', action='store_true', default=config['y']) ns = p.parse_args(['-h']) |
|||
msg400969 - (view) | Author: (Thermi) | Date: 2021-09-02 23:39 | |
1) True. That'd mean such functionality would not be usable by such a workaround though. 2) ANY setting has a default value. The output in the --help message has to, if any defaults at all are shown, be the same as the actual default values. Storing the default values as part of the argparse.ArgumentParser configuration prevents duplication of the default value declaration in the config file reader, and the argument parser. What I request is the reverse of what you wrote. I want the order of priority to fall back to the defaults, if no value is specified in the config file. And if an argument is passed via argv, then that value should take precedence over what is set in the config file. This is in the first message in this issue. 3) Two different places to touch when you want to add a new option: 1) Default config declared in program code 2) argparse.ArgumentParser configuration in code. |
|||
msg400971 - (view) | Author: paul j3 (paul.j3) * | Date: 2021-09-03 00:56 | |
Another way to play with the defaults is to use argparse.SUPPRESS. With such a default, the argument does not appear in the namespace, unless provided by the user. In [2]: p = argparse.ArgumentParser() ...: p.add_argument('--foo', default=argparse.SUPPRESS, help='foo help') ...: p.add_argument('--bar', default='default') ...: p.add_argument('--baz'); In [3]: args = p.parse_args([]) In [4]: args Out[4]: Namespace(bar='default', baz=None) Such a namespace can be used to update an existing dict (such as from a config file), changing only keys provided by user (and ones where SUPPRESS does not make sense, such as store_true and positionals). In [5]: adict = {'foo':'xxx', 'bar':'yyy', 'baz':'zzz'} In [6]: adict.update(vars(args)) In [7]: adict Out[7]: {'foo': 'xxx', 'bar': 'default', 'baz': None} User provided value: In [8]: args = p.parse_args(['--foo','foo','--baz','baz']) In [9]: args Out[9]: Namespace(bar='default', baz='baz', foo='foo') In this code sample I used Ipython. That IDE uses (or at least did some years ago) a custom integration of config and argparse. System default config file(s) set a large number of parameters. Users are encouraged to write their own profile configs (using provided templates). On starting a session, the config is loaded, and used to populate a parser, with arguments, helps and defaults. Thus values are set or reset upto 3 times - default, profile and commandline. I for example, usually start an ipython session with an alias alias inumpy3='ipython3 --pylab qt --nosep --term-title --InteractiveShellApp.pylab_import_all=False --TerminalInteractiveShell.xmode=Plain' Regarding this bug/issue, if someone can come up with a clever tweak that satisfies Thermi, is potentially useful to others, and is clearly backward compatible, great. But if this issue requires a less-than-ideal-compatible patch, or greater integration of config and argparse, then it needs to be developed as a separate project and tested on PyPi. Also search PyPi; someone may have already done the work. |
|||
msg400973 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2021-09-03 01:29 | |
> I want the order of priority to fall back to the defaults, > if no value is specified in the config file. And if an argument > is passed via argv, then that value should take precedence > over what is set in the config file. from collections import ChainMap from argparse import ArgumentParser parser = ArgumentParser() missing = object() for arg in 'abcde': parser.add_argument(f'-{arg}', default=missing) system_defaults = {'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5} config_file = {'a': 6, 'c': 7, 'e': 8} command_line = vars(parser.parse_args('-a 8 -b 9'.split())) command_line = {k: v for k, v in command_line.items() if v is not missing} combined = ChainMap(command_line, config_file, system_defaults) print(dict(combined)) > This is in the first message in this issue. The feature request is clear. What problem you're trying to solve isn't clear. What you're looking for is likely some permutation of the above code or setting a argument default to a value in the ChainMap. I think you're ignoring that we already have ways to set default values to anything that is needed and we already have ways to tell is an argument was not encountered (but not both at the same time). [Paul J3] > So unless someone comes up with a really clever idea, > this is bigger request than it first impressions suggest. I recommend rejecting this feature request. The module is not obliged to be all things to all people. Most variations of the problem already have a solution. We should leave it at that. Extending the namespace with extra boolean arguments would just open a can of worms that would make most users worse off, likely breaking any code that expects the namespace to contain exactly what it already contains. |
|||
msg400986 - (view) | Author: wodny (wodny85) | Date: 2021-09-03 07:39 | |
I used a wrapper to default values. This gives me nice help message with ArgumentDefaultsHelpFormatter and easy way to update a config file dictionary with results from parse_args(). ```python from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter, Namespace from dataclasses import dataclass from typing import Any @dataclass class Default: value: Any def __str__(self): return str(self.value) @staticmethod def remove_defaults(ns): return Namespace(**{ k: v for k, v in ns.__dict__.items() if not isinstance(v, Default)}) @staticmethod def strip_defaults(ns): return Namespace(**{ k: v.value if isinstance(v, Default) else v for k, v in ns.__dict__.items() }) p = ArgumentParser(formatter_class=ArgumentDefaultsHelpFormatter) p.add_argument("--foo", "-f", default=Default(10), help="the foo arg") p.add_argument("--bar", "-b", default=Default("big-bar"), help="the bar arg") p.add_argument("--baz", "-z", default=True, help="the baz arg") options = p.parse_args() print(options) print(Default.remove_defaults(options)) print(Default.strip_defaults(options)) ``` ```sh $ ./arguments.py -b hello Namespace(bar='hello', baz=True, foo=Default(value=10)) Namespace(bar='hello', baz=True) Namespace(bar='hello', baz=True, foo=10) $ ./arguments.py -b hello -h usage: arguments.py [-h] [--foo FOO] [--bar BAR] [--baz BAZ] optional arguments: -h, --help show this help message and exit --foo FOO, -f FOO the foo arg (default: 10) --bar BAR, -b BAR the bar arg (default: big-bar) --baz BAZ, -z BAZ the baz arg (default: True) ``` |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:47 | admin | set | github: 88911 |
2021-09-03 07:39:47 | wodny85 | set | messages: + msg400986 |
2021-09-03 01:29:07 | rhettinger | set | messages: + msg400973 |
2021-09-03 00:56:47 | paul.j3 | set | messages: + msg400971 |
2021-09-02 23:39:52 | Thermi | set | messages: + msg400969 |
2021-09-02 23:28:01 | rhettinger | set | messages: + msg400968 |
2021-09-02 21:07:23 | Thermi | set | messages: + msg400958 |
2021-09-02 20:55:34 | rhettinger | set | messages: + msg400957 |
2021-09-02 15:22:14 | wodny85 | set | nosy:
+ wodny85 |
2021-07-30 20:41:34 | terry.reedy | set | versions:
+ Python 3.11, - Python 3.9 nosy: + terry.reedy messages: + msg398608 components: + Library (Lib), - Tests stage: test needed |
2021-07-29 04:33:20 | paul.j3 | set | messages: + msg398456 |
2021-07-28 19:51:44 | paul.j3 | set | messages: + msg398417 |
2021-07-28 19:47:35 | paul.j3 | set | messages: + msg398416 |
2021-07-28 18:13:06 | shihai1991 | set | nosy:
+ rhettinger, paul.j3 |
2021-07-28 08:55:37 | Thermi | set | messages: + msg398358 |
2021-07-28 07:14:58 | joker | set | versions:
+ Python 3.9, - Python 3.11 nosy: + joker messages: + msg398351 components: + Tests, - Library (Lib) |
2021-07-27 09:26:32 | Thermi | create |