classification
Title: argparse parse_known_args does not work with subparsers
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: bethard Nosy List: bethard, catherine, guandalino, r.david.murray
Priority: normal Keywords: patch

Created on 2010-07-23 12:00 by bethard, last changed 2010-11-02 12:51 by bethard. This issue is now closed.

Files
File name Uploaded Description Edit
test_subparser_parse_known_args.patch catherine, 2010-08-03 17:13
unrecognized_args_defer_exit.patch catherine, 2010-08-04 17:45
Messages (6)
msg111285 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-07-23 12:00
[Moved from http://code.google.com/p/argparse/issues/detail?id=45]

If you try to use parse_known_args and you have a subparser, the subparser will still complain if it sees extra arguments:

>>> parser = argparse.ArgumentParser()
>>> subparsers = parser.add_subparsers()
>>> subparsers.add_parser('A')
>>> parser.parse_known_args(['A', '--foo', 'b'])
usage:  A [-h]
 A: error: unrecognized arguments: --foo b

What should be returned is probably:

>>> parser.parse_known_args(['A', '--foo', 'b'])
(Namespace(), ['--foo', 'b'])

The problem is that subparsers don't know whether they're being called by parse_args() or parse_known_args(). I see a few possible fixes:

* Add another argument to Action.__call__ that indicates what method is being called. But that would break any existing subclasses.

* Do some stack inspection using sys._getframe(). But that's not guaranteed to work on other implementations of Python.

* When parse_args is called, set some flag on the subparsers object that causes it to call parse_known_args instead, and restore that flag before parse_known_args returns. This probably introduces potential threading issues, though practically perhaps they'll never turn up.
msg112629 - (view) Author: Catherine Devlin (catherine) Date: 2010-08-03 16:52
Some basic unit tests for parse_known_args on a subparser.
msg112635 - (view) Author: Catherine Devlin (catherine) Date: 2010-08-03 17:13
Some simple unit tests for parse_known_args on a parser with a subparser.  They are indeed failing on the trunk.
msg112838 - (view) Author: Catherine Devlin (catherine) Date: 2010-08-04 17:45
This patch fixes it with a fourth approach: if unrecognized arguments are found during subparser parsing, information about them is inserted into the namespace (under "._unrecognized"), and the decision about whether to exit is deferred.  When top-level parsing is finished, those recorded unrecognized args are added to whatever was found by the main parser.  

It passes the tests I submitted yesterday.
msg119986 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-10-30 14:03
I've looked over this patch, and it seems to me that there is no compelling reason to create two new functions.  I think it would be clearer to just inline that code in the places it is used.  If it turns out later that the code needs to be reused elsewhere, it can be factored out at that time.  (Also, as it stands the methods are being made part of the public API, and that doesn't look right to me.)

Catherine, if you'd like to update the patch that would be great.  Otherwise we will hopefully get to it during the upcoming bug weekend.
msg120219 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-11-02 12:51
Fixed with a variant of Catherine's patch (following R. David Murray's suggestion of inlining the two methods) in r86111 (3.X) and r86112 (2.7). Also added one more test to make sure that the order of the extra arguments is consistent (extra arguments from main parser before extra arguments from subparser).
History
Date User Action Args
2010-11-02 12:51:08bethardsetstatus: open -> closed
versions: - Python 3.1
messages: + msg120219

resolution: fixed
stage: patch review -> resolved
2010-10-30 14:04:35r.david.murraysetstage: needs patch -> patch review
versions: + Python 3.1, - Python 3.3
2010-10-30 14:03:51r.david.murraysetnosy: + r.david.murray
messages: + msg119986
2010-10-29 17:03:32guandalinosetnosy: + guandalino
2010-08-04 17:45:41catherinesetfiles: + unrecognized_args_defer_exit.patch

messages: + msg112838
2010-08-03 17:13:01catherinesetfiles: + test_subparser_parse_known_args.patch

messages: + msg112635
2010-08-03 17:00:20catherinesetfiles: - test_subparser_parse_known_args.patch
2010-08-03 16:52:27catherinesetfiles: + test_subparser_parse_known_args.patch

nosy: + catherine
messages: + msg112629

keywords: + patch
2010-07-23 12:00:03bethardcreate