classification
Title: Document argparse behaviour when custom namespace object already has the field set
Type: Stage: resolved
Components: Documentation Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: docs@python, miss-islington, paul.j3, rhettinger, zerkms
Priority: normal Keywords: patch

Created on 2019-11-19 02:23 by zerkms, last changed 2020-12-07 05:17 by rhettinger. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 23653 merged rhettinger, 2020-12-05 05:22
PR 23668 merged miss-islington, 2020-12-07 02:29
Messages (7)
msg356939 - (view) Author: Ivan Kurnosov (zerkms) Date: 2019-11-19 02:23
At this moment it's impossible to explain the behaviour of this script using documentation.

Given it was explicitly coded to behave like that - it should be somehow noted in the documentation, that as long as a `CliArgs.foo` field has a default value set already - it won't be overwritten with a default argparse argument value.


```
import argparse

class CliArgs(object):
    foo: str = 'not touched'


parser = argparse.ArgumentParser()
parser.add_argument('--foo', default='bar')

args = CliArgs()
parser.parse_args(namespace=args)
print(args.foo) # 'not touched'

print(parser.parse_args()) # 'bar'
```
msg356943 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2019-11-19 04:34
It doesn't have to be a special class. It can be a `argparse.Namespace` object.  If the preexisting namespace has an attribute set, the action default will not over write it.

    In [85]: parser = argparse.ArgumentParser() 
        ...: parser.add_argument('--foo', default='bar') 
        ...: parser.parse_args([],
                 namespace=argparse.Namespace(foo=123, baz=132))   
    Out[85]: Namespace(baz=132, foo=123)

This is described in comments at the start of parse_known_args()

        ....
        # add any action defaults that aren't present
        for action in self._actions:
            if action.dest is not SUPPRESS:
                if not hasattr(namespace, action.dest):
                    if action.default is not SUPPRESS:
                        setattr(namespace, action.dest, action.default)

        # add any parser defaults that aren't present
        for dest in self._defaults:
            if not hasattr(namespace, dest):
                setattr(namespace, dest, self._defaults[dest])

There are many details about 'defaults' that are not documented.  This might not be the most significant omission.  

I have not seen many questions about the use of a preexisting namespace object (here or on StackOverflow).  While such a namespace can be used to set custom defaults (as shown here), I think it is more useful when using a custom Namespace class, one the defines special behavior.

Originally the main parser's namespace was passed to subparsers.  But a change in 2014, gave the subparser a fresh namespace, and then copied values from it back to the main namespace.  While that gave more power to the subparser's defaults, users lost some ability to use their own namespace class.

https://bugs.python.org/issue27859 - argparse - subparsers does not retain namespace

https://bugs.python.org/issue9351 - argparse set_defaults on subcommands should override top level set_defaults

https://bugs.python.org/issue34827 - Make argparse.NameSpace iterable (closed)
msg356944 - (view) Author: Ivan Kurnosov (zerkms) Date: 2019-11-19 05:34
> I have not seen many questions about the use of a preexisting namespace object (here or on StackOverflow)

as typing was added to the language natively - it should become more and more frequently used.

I personally see no reason anymore to NOT use a custom namespace: typed arguments object is x100 times better than untyped one.
msg357037 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-11-20 08:03
For now, we should at least document that, "If the preexisting namespace has an attribute set, the action default will not over write it."
msg357038 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-11-20 08:06
Ivan, you don't need to specify default values to have typing.  This will suffice:

    class CliArgs(object):
        foo: Optional[str]
        bar: int
        baz: float
msg382617 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-12-07 02:29
New changeset 752cdf21eb2be0a26ea6a34a0de33a458459aead by Raymond Hettinger in branch 'master':
bpo-38843: Document behavior of default when the attribute is already set (GH-23653)
https://github.com/python/cpython/commit/752cdf21eb2be0a26ea6a34a0de33a458459aead
msg382621 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-12-07 05:17
New changeset facca72eae3c0b59b4e89bab81c936dff10fb58a by Miss Islington (bot) in branch '3.9':
bpo-38843: Document behavior of default when the attribute is already set (GH-23653) (#23668)
https://github.com/python/cpython/commit/facca72eae3c0b59b4e89bab81c936dff10fb58a
History
Date User Action Args
2020-12-07 05:17:48rhettingersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-12-07 05:17:17rhettingersetmessages: + msg382621
2020-12-07 02:29:36rhettingersetmessages: + msg382617
2020-12-07 02:29:31miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request22535
2020-12-05 05:22:23rhettingersetkeywords: + patch
stage: patch review
pull_requests: + pull_request22521
2019-11-20 19:15:42rhettingersetassignee: docs@python -> rhettinger
2019-11-20 08:06:33rhettingersetmessages: + msg357038
2019-11-20 08:03:48rhettingersetmessages: + msg357037
2019-11-19 05:34:10zerkmssetmessages: + msg356944
2019-11-19 04:34:59paul.j3setmessages: + msg356943
2019-11-19 02:46:19xtreaksetnosy: + rhettinger, paul.j3
2019-11-19 02:23:43zerkmscreate