classification
Title: argparse: Parser level defaults do not always override argument level defaults
Type: behavior Stage:
Components: Documentation Versions: Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, kop, paul.j3
Priority: normal Keywords:

Created on 2015-03-30 18:40 by kop, last changed 2015-03-31 22:20 by paul.j3.

Messages (2)
msg239632 - (view) Author: Karl O. Pinc (kop) * Date: 2015-03-30 18:40
In the argparse library parser library, contrary to the documentation,
parser-level defaults do not always override argument-level defaults.

https://docs.python.org/3.5/library/argparse.html#argparse.ArgumentParser.set_defaults

says "Note that parser-level defaults always override argument-level defaults:"

(And so does the python 3.3 docs.)

The docs then provide this example:

  >>> parser = argparse.ArgumentParser()
  >>> parser.add_argument('--foo', default='bar')
  >>> parser.set_defaults(foo='spam')
  >>> parser.parse_args([])
  Namespace(foo='spam')

But it is only true that parser-level defaults override argument-level
defaults when they are established after the argument is added.

The output below shows an argument level default overrideing
a parser level default.

$ python3
Python 3.3.2 (default, Jun  4 2014, 11:36:37) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.set_defaults(foo='spam')
>>> parser.add_argument('--foo', default='bar')
_StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default='bar', type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args([])
Namespace(foo='bar')

It seems that whichever default is set last is the one which is used.
Or perhaps there are not argument level defaults and parser level
defaults, there are just defaults, period.  (It might, possibly,
be nice if there _were_ both argument and parser level defaults
and parser level defaults had priority.  Then this would not be
a documentation bug.)
msg239754 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2015-03-31 22:20
The handling of defaults is a bit complicated.

Note that `set_defaults` both sets a `_defaults` attribute, and any actions with a matching `dest`.  So it could change the `default` of 0, 1 or more actions.

    def set_defaults(self, **kwargs):
        self._defaults.update(kwargs)

        # if these defaults match any existing arguments, replace
        # the previous default on the object with the new one
        for action in self._actions:
            if action.dest in kwargs:
                action.default = kwargs[action.dest]


`add_argument` gets `default` from the default parameter (kwarg), the container's `_defaults` or the parser `.argument_default`.  The earliest in that list has priority.

Finally, at the start of `parse_known_args`, each action's `default` is added to the namespace (IF it isn't already there), and values from `_defaults` are also add (also IF not already present).  So here the priority is user supplied `namespace`, action `default`, parser `_defaults`.

And finally, at the end of `_parse_known_args`, any string values in the namespace that match their action default are passed through `get_values`, which may convert them via the 'type'.

I've skimmed over a couple of things:

- defaults defined by the Action class (e.g. 'store_true' sets a False default)

- how parent's `_defaults` are copied

- when `_defaults` affects that final `get_values` action.

- the relative priority of parser and subparsers _defaults (a current bug issue).

So as you observed, `set_defaults` can change a previously defined argument's `default`, but it does not have priority over parameters supplied to new arguments.

In my opinion, `set_default` is most useful as a way of setting Namespace values that do not have their own argument.  The documentation has an example where each subparser sets its own 'func' attribute.

    parser_foo.set_defaults(func=foo)

Maybe the documentation could be refined, though it might be tricky to do so without adding further confusion.

Changing the priorities is probably not a good idea.  The recent bug issues about parser and subparser `set_defaults` a cautionary tale.  Even the earlier change in how 'get_values' is applied to defaults has potential pitfalls.
History
Date User Action Args
2015-03-31 22:20:23paul.j3setnosy: + paul.j3
messages: + msg239754
2015-03-30 18:40:47kopcreate