Title: argparse does not honor prefix_chars when adding default options
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 2.7
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bethard, catherine, doughellmann, r.david.murray, ted.turocy
Priority: normal Keywords: patch

Created on 2010-07-31 22:20 by doughellmann, last changed 2010-08-03 19:44 by doughellmann. This issue is now closed.

File name Uploaded Description Edit doughellmann, 2010-07-31 22:20
argparse_char_fix+doc.patch ted.turocy, 2010-08-01 23:55 patch against py3k to fix bug and make docs more clear
argparse_test.patch catherine, 2010-08-02 02:30 unit tests
Messages (20)
msg112219 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-07-31 22:20
If an ArgumentParser is created with a prefix_chars string that does not include '-', the default options created for showing help (-h and --help) and the version (-v and --version) are invalid and generate an exception.
msg112220 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-07-31 22:21
One solution would be to use the first character of prefix_chars when building those default options.
msg112257 - (view) Author: Theodore Turocy (ted.turocy) Date: 2010-08-01 02:09
What is the appropriate behavior for long options when '-' is not one of the accepted prefix_chars?  All of '-h', '--help', '-v', and '--version' are hardcoded in the ctor.  If, for instance, prefix_chars='+', should the long options be '++help', or should long options simply be disabled?

Also, for backwards compatibility, to implement Doug Hellmann's suggestion, '-' should be used for '-h' and '--help' whenever '-' is listed in prefix_chars (so, e.g., prefix_chars='+-' will result in the same behavior as previous versions). 

I have a working patch for this but wouldn't mind someone else's thoughts first.
msg112258 - (view) Author: Theodore Turocy (ted.turocy) Date: 2010-08-01 02:40
Looking at the test fixtures that exercise argparse, it appears that the intended behavior when '-' is not a prefix_char is to accept a doubling of any of the prefix_chars for long arguments.  That is, if '-' is not present in prefix_chars but ':' is, then an argument like '::help' would be a valid long argument.

I'm attaching a patch which fixes the originally reported problem as follows:

* If '-' is in prefix_chars, then the help and version arguments are '-h', '--help', '-v', '--version'
* If '-' is not in prefix_chars, then the first char in prefix_chars is used to lead the option, e.g., '+h', '++help', '+v', '++version' if prefix_chars starts with '+'.

Catherine Devlin is also sprinting here at PyOhio and will have a test fixture separately.
msg112308 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-08-01 10:44
Yes, this looks fine, assuming a test is also added.
msg112316 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-08-01 11:55
I haven't read the existing tests, but I am not seeing the behavior described by Ted in msg112258.  If I specify the prefix_chars as '+/' and define a long option '//myopt' then using ++myopt on the command line gives an error that the option is unrecognized.  I don't know if that's a bug or the desired behavior.
msg112317 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-08-01 11:56
Oh, I should point out that last comment is describing what I see when using the unpatched 2.7 version of the module.
msg112326 - (view) Author: Theodore Turocy (ted.turocy) Date: 2010-08-01 13:54
I was less than clear in what I wrote last night, Doug.  What I mean is that the idiom "::foo" for a long argument, instead of the more standard-looking "--foo", appears in the test suite.  This suggests to me that the intended behavior for the default help option should be to use a doubled prefix character in front of the long-form option.

For instance, if prefix_chars='+' and add_help=True, then the automatically provided help arguments will be '+h' and '++help'.

Certainly, if one explicitly adds an option '//myopt', we would not expect '::myopt' to also be accepted.  I'm talking here only about the options that argparse adds as a convenience.
msg112330 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-08-01 14:05
I was actually surprised that prefix_chars didn't allow *any* of those characters to indicate an option.  For example, a program on Unix might use options that start with '-', but also support '/' as a prefix under Windows.  If that's the intended behavior, that's OK, and maybe the docs can be made more specific.

It does, however, open the question of how to pick the prefix to use for automatically-generated options.  Maybe the current behavior is best after all, so the programmer disabling the '-' prefix character realizes they need to handle the help and version options explicitly.
msg112353 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-08-01 16:46
It is intentional that you have to specify both "/foo" and "+foo" if you want them to be aliases for the same argument. A common use case for prefix_chars is to define "+x" and "-x" options, which usually mean different things.

As far as the current issue, I believe the current behavior is buggy - an argument that you can never use is added to the parser automatically:

>>> parser = argparse.ArgumentParser(prefix_chars='+')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bethard/Documents/projects/python/2.X/Lib/", line 1569, in __init__
    help=_('show this help message and exit'))
  File "/Users/bethard/Documents/projects/python/2.X/Lib/", line 1246, in add_argument
    kwargs = self._get_optional_kwargs(*args, **kwargs)
  File "/Users/bethard/Documents/projects/python/2.X/Lib/", line 1369, in _get_optional_kwargs
    raise ValueError(msg % tup)
ValueError: invalid option string '-h': must start with a character '+'

Yes you could explicitly specify add_help=True, but I think using an appropriate prefix character is actually the right behavior.
msg112355 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-08-01 16:51
Explicitly specifying aliases makes sense, it just wasn't clear that was the intent from the existing documentation.  So, I don't think the behavior needs to change, but a doc update might help.
msg112356 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-08-01 16:55
A doc patch would also be welcome, but I do think it's a bug that "ArgumentParser(prefix_chars='+')" throws an exception, and I think Ted's patch looks fine to me.
msg112357 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-08-01 17:01
Sorry I'm not being clear: I do like the patch, I think the exception should not be raised.
msg112406 - (view) Author: Theodore Turocy (ted.turocy) Date: 2010-08-01 23:55
I'm uploading a new version of my patch which includes a proposed clarification to the documentation about the behavior in this case.  Doug, does this make the documentation clearer to you?  It is now explicit about the behavior for formulating the help option if '-' is not in prefix_chars.
msg112445 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-08-02 11:50
Yes, that doc change  is clear.  Thanks!
msg112472 - (view) Author: Catherine Devlin (catherine) Date: 2010-08-02 14:07
Attached a unit test patch corresponding to Ted's patch.
msg112555 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-08-02 21:44
The combined patches look good to me, and I tested the tests and patch.  (I accidentally deleted the earlier fix patch, sorry).  It would be nice to add one more test case that does prefix="+-/" to make sure - is used when it isn't first.

Steven, I'd be happy to commit this if you are OK with it, since I was mentoring the people who wrote it.
msg112603 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-08-03 13:20
Yep, I'm fine with you committing this (after adding the prefix="+-/" you suggested). I don't have time right now to test the patches, but the code looks about right, and the tests ran fine for you, so I'm fine with it.
msg112644 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-08-03 18:14
Committed (with the additional test) to py3k in r83657, and 2.7 in r83676.
msg112657 - (view) Author: Doug Hellmann (doughellmann) * (Python committer) Date: 2010-08-03 19:44
Thanks, everyone!
Date User Action Args
2010-08-03 19:44:41doughellmannsetmessages: + msg112657
2010-08-03 18:14:34r.david.murraysetstatus: open -> closed

messages: + msg112644
stage: commit review -> resolved
2010-08-03 13:20:19bethardsetmessages: + msg112603
2010-08-02 21:44:09r.david.murraysetnosy: + r.david.murray

messages: + msg112555
stage: patch review -> commit review
2010-08-02 21:02:28r.david.murraysetfiles: - argparse_char_fix.patch
2010-08-02 20:36:30r.david.murraysetstage: test needed -> patch review
2010-08-02 14:07:27catherinesetnosy: + catherine
messages: + msg112472
2010-08-02 11:50:51doughellmannsetnosy: + ted.turocy
messages: + msg112445
2010-08-02 02:30:57catherinesetfiles: + argparse_test.patch
nosy: - ted.turocy
2010-08-01 23:55:37ted.turocysetfiles: + argparse_char_fix+doc.patch

messages: + msg112406
2010-08-01 17:01:51doughellmannsetmessages: + msg112357
2010-08-01 16:55:03bethardsetmessages: + msg112356
2010-08-01 16:51:00doughellmannsetmessages: + msg112355
2010-08-01 16:46:22bethardsetmessages: + msg112353
2010-08-01 14:05:40doughellmannsetmessages: + msg112330
2010-08-01 13:54:14ted.turocysetmessages: + msg112326
2010-08-01 11:56:23doughellmannsetmessages: + msg112317
2010-08-01 11:55:35doughellmannsetmessages: + msg112316
2010-08-01 10:44:30bethardsetmessages: + msg112308
2010-08-01 02:40:18ted.turocysetfiles: + argparse_char_fix.patch
keywords: + patch
messages: + msg112258
2010-08-01 02:09:09ted.turocysetnosy: + ted.turocy
messages: + msg112257
2010-08-01 00:52:16r.david.murraysetnosy: + bethard
stage: test needed
type: behavior

versions: + Python 3.2
2010-07-31 22:21:13doughellmannsetmessages: + msg112220
2010-07-31 22:20:26doughellmanncreate