classification
Title: ArgumentParser inconsistent with parse_known_args
Type: behavior Stage: needs patch
Components: Library (Lib) Versions: Python 3.3, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Andrei.Vereha, Omar Sandoval, Radu.Ciorba, Sam.Breese, bethard, idank, kynikos, paul.j3, r.david.murray
Priority: normal Keywords: easy, patch

Created on 2012-10-05 14:53 by idank, last changed 2016-06-30 18:50 by Omar Sandoval.

Files
File name Uploaded Description Edit
aparse.py idank, 2012-10-05 14:53
dashku.patch paul.j3, 2013-04-17 22:38 review
dashku.patch paul.j3, 2013-04-29 07:01 review
Messages (9)
msg172088 - (view) Author: Idan Kamara (idank) Date: 2012-10-05 14:53
When known and unknown options are given together in the same option string (e.g. -xy) then ArgumentParser behaves in a strange way:

- if the known option is given first (so -k is known and the parser is fed with ['-ku']) then the parsing aborts with "error: argument -k/--known: ignored explicit argument 'u'"

- if the unknown option is given first then both options are treated as unknown and returned in the list of remaining arguments.

This makes it impossible to use parse_known_args for its intended purpose because every single letter option might be interleaved with other unknown options.

I attached a test script that demonstrates this.
msg172089 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-10-05 15:03
Looks like parse_known_args needs to be taught to not treat unknown text following an option as an argument if the option does not take an argument.  That would be in keeping with its mission, I think :)

There will still probably be ambiguous cases that will trip you up, but that should help.
msg172091 - (view) Author: Idan Kamara (idank) Date: 2012-10-05 15:19
Yes that'd fix the known option before unknown but not the other way around.
msg172092 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-10-05 15:43
Right.  I didn't read what you wrote carefully enough.  Clearly parse_known_args is buggy here.
msg173492 - (view) Author: Sam Breese (Sam.Breese) Date: 2012-10-22 00:32
Writing a patch now. Should be ready in a few hours.
msg173640 - (view) Author: Sam Breese (Sam.Breese) Date: 2012-10-23 22:14
Make that a few days. I fixed the case where the known arg is first, but not the other one. Will get to it soon, hopefully.
msg181167 - (view) Author: Radu Ciorba (Radu.Ciorba) Date: 2013-02-02 13:37
"- if the unknown option is given first then both options are treated as unknown and returned in the list of remaining arguments."
I don't think this case is correct behaviour. There is no way to determine if u accepts arguments or not. If nargs for -u is "?" or more, the current behavior is correct since everything after -u would be an argument for this option.
msg187212 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-04-17 22:38
parser = argparse.ArgumentParser()
    parser.add_argument('-k','--known',action='store_true')
    print(parser.parse_known_args(['-k','-u']))
    print(parser.parse_known_args(['-ku']))
    print(parser.parse_known_args(['-uk']))

I think you want these 3 cases to produce the same output:

    (Namespace(known=True), ['-u'])

With the attached patch, '-ku' produces the same result as '-k -u'. Instead of raising the "ignored explicit argument 'u'" error, if puts '-u' in the 'extras' list.  'parse_args' then raises a "error: unrecognized arguments: u" error.

That's an easy change, and does not break anything in the 'test_argparse.py' file.  But keep in mind that this test file mostly uses 'parse_args', and usually it ignores the failure messages.

Getting '-uk' to work this way would be much harder.  While it isn't obvious from the documentation, '-uk' is a valid option string, and '-u' is a valid abbreviation. Notice in 16.4.4.1. of the documentation, the difference between long and short options is based on the number of characters, not whether there it starts with '-' or '--'.  So identifying what is wrong with '-uk' would require ambiguous reasoning.

I wonder what optparse does.
msg188038 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-04-29 07:01
Correction: The patch I gave in the last message produces:

    >>> parser.parse_known_args(['-ku'])
    (Namespace(known=False), ['u'])

It doesn't take action on the '-k', and puts 'u' in extras, not '-u'.

This new patch gets it right:

    >>> parser.parse_known_args(['-ku'])
    (Namespace(known=True), ['-u'])

We need more test cases, including ones that work as expected with optparse or other unix parsers.
History
Date User Action Args
2016-06-30 18:50:07Omar Sandovalsetnosy: + Omar Sandoval
2016-03-19 03:57:38kynikossetnosy: + kynikos
2013-04-29 07:01:55paul.j3setfiles: + dashku.patch

messages: + msg188038
2013-04-17 22:38:07paul.j3setfiles: + dashku.patch

nosy: + paul.j3
messages: + msg187212

keywords: + patch
2013-02-02 13:42:02Andrei.Verehasetnosy: + Andrei.Vereha
2013-02-02 13:37:18Radu.Ciorbasetnosy: + Radu.Ciorba
messages: + msg181167
2012-10-23 22:14:48Sam.Breesesetmessages: + msg173640
2012-10-22 00:32:51Sam.Breesesetnosy: + Sam.Breese
messages: + msg173492
2012-10-05 15:43:36r.david.murraysetmessages: + msg172092
2012-10-05 15:19:21idanksetmessages: + msg172091
2012-10-05 15:03:02r.david.murraysetnosy: + r.david.murray, bethard
messages: + msg172089

keywords: + easy
stage: needs patch
2012-10-05 14:53:21idankcreate