classification
Title: argparse - subparsers does not retain namespace
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: argparse set_defaults on subcommands should override top level set_defaults
View: 9351
Assigned To: Nosy List: Mariatta, ericvw, nickpapior, paul.j3
Priority: normal Keywords:

Created on 2016-08-25 15:02 by nickpapior, last changed 2016-10-06 15:46 by ned.deily. This issue is now closed.

Files
File name Uploaded Description Edit
argparse_subspacenamespace.py nickpapior, 2016-08-25 15:02 Example showing the crash of the argparse failure.
issue27859test.py paul.j3, 2016-09-04 16:36
Messages (5)
msg273654 - (view) Author: Nick Papior (nickpapior) Date: 2016-08-25 15:02
Using argparse does not retain the Namespace variable across sub-parsers.

This prohibits customization of Actions due to "upper" level arguments not being stored in the namespace passed to the sub-parsers.

Hence, one may not create different sub-parsers which depend on an upper level flag.

The reason is the subparser command:
    subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)

Which by default passes `None` as the namespace argument.

A simple patch would be to copy the original namespace, which may then be altered. And subsequently restored, based on changes to that namespace.
msg274337 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2016-09-04 04:54
This call used to be

    namespace, arg_strings = parser.parse_known_args(arg_strings, namespace)

But in 2014 (2.7.9) http://bugs.python.org/issue9351 was implemented

As noted in the title and comment in the code, the idea was to give more power to the defaults set in the subparser.

But as noted in the subsequent posts, the patch has raised some backward compatibility issues.

Generally having the action for one argument depend on the value of another argument is tricky, because arguments may be parsed in any order. (but there is a 'test_argparse.py' case that does that kind of testing).  It is safer to test for any interactions after parsing is all done.

Now in the subparser case, there is an order.  Main parser arguments have to come first.

I suggested in 9351 a way of allowing both behaviors - use of new namespace or use of the existing one.  But as you can see there hasn't been any further action in the past 2 years.
msg274368 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2016-09-04 16:36
I've posted a file that runs your code as you expect.

It uses a custom Action class (like your test case).  It subclasses ._SubParsersAction, and replaces the 9351 namespace use with the original one.  I use the registry to change the class that parser.add_subparsers() uses.

The stock argparse.py file does not need to be changed.

ps

In your custom Action I access `namespace.foo` with `getattr(namespace, 'foo', None)` which is how argparse accesses the namespace, and does not throw attribute errors.
msg277936 - (view) Author: Nick Papior (nickpapior) Date: 2016-10-03 08:23
Sorry I haven't responded previously.

Thanks a lot for helping me. I hadn't realized the `register` function.
msg278173 - (view) Author: Mariatta (Mariatta) * (Python committer) Date: 2016-10-06 05:14
Seems like this can be closed.
Thanks.
History
Date User Action Args
2016-10-06 15:46:59ned.deilysetstatus: open -> closed
superseder: argparse set_defaults on subcommands should override top level set_defaults
resolution: duplicate
stage: needs patch -> resolved
2016-10-06 05:14:50Mariattasetnosy: + Mariatta
messages: + msg278173
2016-10-03 08:23:32nickpapiorsetmessages: + msg277936
2016-09-04 16:36:55paul.j3setfiles: + issue27859test.py

messages: + msg274368
2016-09-04 04:54:41paul.j3setnosy: + paul.j3
messages: + msg274337
2016-08-25 17:30:16ericvwsetnosy: + ericvw
2016-08-25 16:15:07SilentGhostsetstage: needs patch
versions: - Python 3.2, Python 3.3, Python 3.4
2016-08-25 15:02:10nickpapiorcreate