msg156254 - (view) |
Author: Michele Orrù (maker) * |
Date: 2012-03-18 14:29 |
http://docs.python.org/library/argparse.html#arguments-containing
The attached file shows different behaviours when using '--' immediately after an optional argument.
tumbolandia:cpython maker$ python foo.py --test=-- foo
[]
tumbolandia:cpython maker$ python foo.py --test -- foo
usage: foo.py [-h] [-t TEST] [yuri]
foo.py: error: argument -t/--test: expected 1 argument(s)
The same is for single-dash arguments.
tumbolandia:cpython maker$ python foo.py -t -- foo
usage: foo.py [-h] [-t TEST] [yuri]
foo.py: error: argument -t/--test: expected 1 argument(s)
tumbolandia:cpython maker$ python foo.py -t-- foo
[]
Obviously argparse should return an error in both cases.
The bug is probably due to Lib/argparser.py:2211
|
msg156256 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2012-03-18 14:45 |
It does look like there's anomalous behavior here of some sort, but I'd expect --test=-- to result in test="--", myself, rather than an error. My intuition is that '--' would need to be preceded by a space to function as the 'end of options' marker. Because of that, I've never tried the above scenario with a unix command, so maybe my intuition is wrong :)
Just for reference (the code paths may be different), there is another open issue about -- parsing, issue 13922.
|
msg156275 - (view) |
Author: Steven Bethard (bethard) * |
Date: 2012-03-18 21:31 |
I just tried this with grep's "-e" and "--regexp":
$ cat > temp.txt
a--b
cdef
$ grep -e-- -v temp.txt
cdef
$ grep --regexp=-- -v temp.txt
cdef
$ grep -e -- -v temp.txt
cdef
$ grep --regexp -- -v temp.txt
cdef
And with diff's "-I" and "--ignore-matching-lines":
$ cat > temp2.txt
cdef
a--b
$ diff temp.txt temp2.txt
1d0
< a--b
2a2
> a--b
$ diff -I-- temp.txt temp2.txt
$ diff -I -- temp.txt temp2.txt
$ diff --ignore-matching-lines -- temp.txt temp2.txt
$ diff --ignore-matching-lines=-- temp.txt temp2.txt
$
Note though that for options that don't take an argument, the "--" is just removed:
$ grep -v -- a temp.txt
cdef
$ diff -i -- temp.txt temp2.txt
1d0
< a--b
2a2
> a--b
So I guess the unix rule is: if an option that takes an argument is followed by "--", use that as the option's argument and continue parsing as usual. If an option that takes no argument is followed by "--", then delete the "--" and treat all following flags as positional arguments.
Argparse can't follow this directly, because then people who are using "--" to signal the end of an option with nargs="*" would start getting "--" included in that list. (And I know people have used "--" this way for a while.)
I guess my preference is what R. David Murray suggests: "--" when part of an argument (i.e. not separated by spaces) is treated like any other characters, and only a lone "--" signals the end of options (and is ignored otherwise). That would mean that both "--test=--" and "-t--" in your example would give you ["--"], and the other errors would stay as you saw them.
|
msg156293 - (view) |
Author: Michele Orrù (maker) * |
Date: 2012-03-18 23:40 |
+1 also for me.
I will try to work for a patch in the next days. :)
|
msg187283 - (view) |
Author: Mark Lawrence (BreamoreBoy) * |
Date: 2013-04-18 19:27 |
@Michele could you provide a patch for this?
|
msg187344 - (view) |
Author: paul j3 (paul.j3) * |
Date: 2013-04-19 06:20 |
The patch that I recently submitted for http://bugs.python.org/issue13922
appears to solve this issue. It only removes the '--' that marked the end of options.
With:
parser = argparse.ArgumentParser()
parser.add_argument('-f','--foo')
print(parser.parse_args(['-f--']))
print(parser.parse_args(['--foo=--']))
print(parser.parse_args(['-f', '--']))
I get:
Namespace(foo='--')
Namespace(foo='--')
usage: foodash.py [-h] [-f FOO]
foodash.py: error: argument -f/--foo: expected one argument
|
msg187346 - (view) |
Author: Michele Orrù (maker) * |
Date: 2013-04-19 06:51 |
wow, I was just writing the unittests, thanks paul.
Shall I continue? I don't see any test case on tip.
|
msg187349 - (view) |
Author: Michele Orrù (maker) * |
Date: 2013-04-19 08:12 |
Yes, http://bugs.python.org/file29845/dbldash.patch seems to fix this.
Attaching the unittests, and noisying on your issue.
|
msg262545 - (view) |
Author: Martin Panter (martin.panter) * |
Date: 2016-03-28 01:49 |
In Python 3.5, this does not seem fixed. Issue 13922 is marked as resolved and fixed, but Pauls’s patch has not actually been committed.
>>> ap = ArgumentParser()
>>> ap.add_argument("-t", "--test", type=str, nargs=1)
_StoreAction(option_strings=['-t', '--test'], dest='test', nargs=1, const=None, default=None, type=<class 'str'>, choices=None, help=None, metavar=None)
>>> ap.add_argument('yuri', nargs='?')
_StoreAction(option_strings=[], dest='yuri', nargs='?', const=None, default=None, type=None, choices=None, help=None, metavar=None)
>>> ap.parse_args(["--test=--", "foo"])
Namespace(test=[], yuri='foo')
>>> ap.parse_args(["--test", "--", "foo"])
usage: [-h] [-t TEST] [yuri]
: error: argument -t/--test: expected 1 argument
argparse.ArgumentError: argument -t/--test: expected 1 argument
During handling of the above exception, another exception occurred:
__main__.SystemExit: 2
>>> ap.parse_args(["-t--", "foo"])
Namespace(test=[], yuri='foo')
>>> ap.parse_args(["-t", "--", "foo"])
usage: [-h] [-t TEST] [yuri]
: error: argument -t/--test: expected 1 argument
argparse.ArgumentError: argument -t/--test: expected 1 argument
During handling of the above exception, another exception occurred:
__main__.SystemExit: 2
|
msg262558 - (view) |
Author: paul j3 (paul.j3) * |
Date: 2016-03-28 05:55 |
I made a mistake of trying to add to or refine a closed patch. Maybe I need to move the dbldash.patch over here for more formal consideration.
|
msg283501 - (view) |
Author: paul j3 (paul.j3) * |
Date: 2016-12-17 17:52 |
I've copied 'dbldash.patch' over from http://bugs.python.org/issue13922. I mistakenly added it to a closed issue.
The patch is old, and should be revised. It is also missing tests for this '--opt=--' case, even though it solves it.
I also reference this patch in http://bugs.python.org/issue9571, which tries to refine dbldash handling for PARSER, REMAINDER nargs.
I'm raising the priority on this because the issue has a come up a couple of times on Stackoverflow:
http://stackoverflow.com/questions/40685320/python-argparse-with-as-the-value
|
msg351230 - (view) |
Author: Hai Shi (shihai1991) * |
Date: 2019-09-06 05:44 |
some test cases which paul provided looks doesn't keep compatible.
In TestDoubleDashRemoval:
# output in my env is Namespace(cmd='cmd', foo=None, rest=['--', '--foo'])
('-- cmd -- -- --foo', NS(cmd='cmd', foo=None, rest=['--', '--', '--foo']))
# output in my env is Namespace(cmd='--', foo='1', rest=['1', '2'])
('-f1 -- -- 1 -- 2', NS(cmd='--', foo='1', rest=['1', '--', '2']))
# output in my env is Namespace(cmd='--foo', foo=None, rest=['--bar', '2'])
('-- --foo -- --bar 2', NS(cmd='--foo', foo=None, rest=['--', '--bar', '2']))
# output in my env is Namespace(cmd='cmd', foo=None, rest=['--foo', '-f2'])
('cmd -- --foo -- -f2', NS(cmd='cmd', foo=None, rest=['--foo', '--', '-f2']))
In TestDoubleDashRemoval1:
# output in my env is Namespace(cmd='cmd', foo='1', rest=['-f2', '3'])
('-f1 -- cmd -- -f2 3', NS(cmd='cmd', foo='1', rest=['--', '-f2', '3']))
my python's version is: 2.7.5, 3.6.8
|
msg353865 - (view) |
Author: Jonas Schäfer (jssfr) |
Date: 2019-10-03 16:19 |
Since I have been adversely affected by this bug ([1]), I looked at the patches.
I combined issue14364.test.patch (which adds test cases for --foo=--) and dbldash.patch in my local working tree and that seems to resolve the issue (tests pass if and only if I apply dbldash.patch and my reproducer from [1]) passes too).
The patches do not contain any type of metainformation, so I’m not comfortable with submitting this as a PR. I am also not at all familiar with the process of managing the changelog etc.
This should serve as a confirmation that this issue can be resolved with the patches. Someone more familiar than me with the process could take up the task to get this merged.
Note: As you can see in [1], this issue has already caused data loss.
[1]: https://github.com/borgbackup/borg/issues/4769
|
msg354001 - (view) |
Author: Hai Shi (shihai1991) * |
Date: 2019-10-05 10:43 |
Hi, Jonas. Thanks for your report.
Let us wait core team's discuss;)
|
msg354049 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2019-10-06 21:22 |
Paul and David, is the current PR complete in your opinion? Are there known cases where the PR would break existing, working code?
Also there needs to be doc entry that is clear on both new and old strategy for distinguishing options from arguments when a dash or dashdash is present. The PR's Misc/NEWS entry, "Fix behavior of argparse when '--' as argument to option", is insufficient.
|
msg354066 - (view) |
Author: Hai Shi (shihai1991) * |
Date: 2019-10-07 08:54 |
> The PR's Misc/NEWS entry, "Fix behavior of argparse when '--' as argument to option", is insufficient.
Thanks for your comment, Raymond. I would continue update the desc.
|
msg389361 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2021-03-23 01:07 |
Unassigning because I haven't been able to make time for this one.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:57:28 | admin | set | github: 58572 |
2021-03-23 01:07:11 | rhettinger | set | assignee: rhettinger -> messages:
+ msg389361 |
2019-10-07 08:54:41 | shihai1991 | set | messages:
+ msg354066 |
2019-10-06 21:22:36 | rhettinger | set | assignee: rhettinger messages:
+ msg354049 versions:
+ Python 3.8, Python 3.9, - Python 3.6 |
2019-10-05 10:43:35 | shihai1991 | set | messages:
+ msg354001 |
2019-10-03 16:19:59 | jssfr | set | nosy:
+ jssfr messages:
+ msg353865
|
2019-09-06 05:58:28 | shihai1991 | set | nosy:
+ rhettinger type: behavior
|
2019-09-06 05:58:00 | shihai1991 | set | stage: needs patch -> patch review pull_requests:
+ pull_request15369 |
2019-09-06 05:44:55 | shihai1991 | set | nosy:
+ shihai1991 messages:
+ msg351230
|
2016-12-17 17:52:20 | paul.j3 | set | files:
+ dbldash.patch priority: normal -> high versions:
+ Python 3.6, Python 3.7, - Python 3.2, Python 3.3 messages:
+ msg283501
keywords:
+ needs review |
2016-03-28 05:55:28 | paul.j3 | set | messages:
+ msg262558 |
2016-03-28 01:49:32 | martin.panter | set | nosy:
+ martin.panter
messages:
+ msg262545 title: Argparse incorrectly handles '--' -> Argparse incorrectly handles '--' as argument to option |
2014-02-03 18:32:34 | BreamoreBoy | set | nosy:
- BreamoreBoy
|
2013-04-19 08:12:09 | maker | set | files:
+ issue14364.test.patch keywords:
+ patch messages:
+ msg187349
|
2013-04-19 06:51:42 | maker | set | messages:
+ msg187346 |
2013-04-19 06:20:51 | paul.j3 | set | messages:
+ msg187344 |
2013-04-19 05:09:46 | paul.j3 | set | nosy:
+ paul.j3
|
2013-04-18 19:27:49 | BreamoreBoy | set | nosy:
+ BreamoreBoy messages:
+ msg187283
|
2012-03-24 15:21:54 | eric.araujo | set | nosy:
+ eric.araujo
|
2012-03-18 23:40:46 | maker | set | messages:
+ msg156293 |
2012-03-18 21:31:48 | bethard | set | messages:
+ msg156275 |
2012-03-18 14:45:00 | r.david.murray | set | versions:
+ Python 3.2, Python 3.3, - Python 3.1 nosy:
+ r.david.murray, bethard
messages:
+ msg156256
components:
+ Library (Lib) stage: needs patch |
2012-03-18 14:29:12 | maker | create | |