classification
Title: argparse: using parents & subcommands, options can be ignored
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Lucas Cimon, paul.j3
Priority: normal Keywords: patch

Created on 2021-12-16 15:29 by Lucas Cimon, last changed 2022-01-10 18:55 by paul.j3.

Pull Requests
URL Status Linked Edit
PR 30146 open Lucas Cimon, 2021-12-16 15:32
Messages (3)
msg408711 - (view) Author: Lucas Cimon (Lucas Cimon) * Date: 2021-12-16 15:29
Hi!

Here is some minimal code reproducing the issue:

    import argparse

    common_opts_parser = argparse.ArgumentParser(add_help=False)
    common_opts_parser.add_argument("--endpoint", choices=("prod", "dev"), default="dev")

    parser = argparse.ArgumentParser(parents=[common_opts_parser])
    subparsers = parser.add_subparsers(required=True)

    subcmd_cmd = subparsers.add_parser("subcmd", parents=[common_opts_parser])
    subcmd_cmd.add_argument("--debug", action="store_true")

    print(parser.parse_args())

Everything works fine / as expected when specifying the common optional arg last:

    $ ./bug_repro.py subcmd --endpoint=dev
    Namespace(endpoint='dev', debug=False)

However when specifying the --endpoint between the program and the subcommand, the value provided is ignored:

    $ ./bug_repro.py --endpoint=dev subcmd
    Namespace(endpoint=None, debug=False)

I have a PR ready to fix that.
msg408730 - (view) Author: Lucas Cimon (Lucas Cimon) * Date: 2021-12-16 19:31
The GitHub PR is ready for reviewing.
msg410241 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2022-01-10 18:55
This patch should be rejected.

By using `common_opts_parser` as parent to both the main and subparsers, you have defined the same argument in both.

By a long standing patch, the value assigned in the subparser has precedence (whether it's the default or user supplied).

https://bugs.python.org/issue9351
argparse set_defaults on subcommands should override top level set_defaults

Partial retraction of this override has been explored, but any changes are still up for debate

https://bugs.python.org/issue45235
argparse does not preserve namespace with subparser defaults

The bottom line is that we should not try to define the same argument (or more specifically the same `dest`) in both main and subparsers.  Duplicate flags are ok.  

But resolving the conflicting values should be done after parsing.  Some users will want to give priority to the main parser's values, others to the subparser's.  Or their own defaults.  argparse cannot handle all cases cleanly.
History
Date User Action Args
2022-01-10 18:55:16paul.j3setmessages: + msg410241
2022-01-02 01:29:22paul.j3setnosy: + paul.j3
2021-12-16 19:31:43Lucas Cimonsetmessages: + msg408730
2021-12-16 15:32:13Lucas Cimonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request28364
2021-12-16 15:29:06Lucas Cimoncreate