classification
Title: argparse does not handle arguments correctly after --
Type: security Stage:
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: invalid
Dependencies: Superseder:
Assigned To: bethard Nosy List: bethard, eric.smith, georg.brandl, iElectric, r.david.murray
Priority: normal Keywords:

Created on 2010-06-24 23:34 by iElectric, last changed 2010-06-29 07:12 by georg.brandl. This issue is now closed.

Messages (8)
msg108567 - (view) Author: Domen Kožar (iElectric) Date: 2010-06-24 23:34
Sample code:

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('-v', action='store_true')
>>> parser.add_argument('foo')
>>> parser.parse_args(['-v', '--', '--foo'])
Namespace(foo='--foo', v=True)

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('-v', action='store_true')
>>> parser.parse_args(['--', '-f'])
usage: [-h] [-v]
: error: unrecognized arguments: -- -f
msg108591 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-06-25 12:25
Not a bug -- you have to tell argparse that you expect a positional argument.  Try without the '--' and an argument that doesn't start with '-', and it will still give you that error.
msg108598 - (view) Author: Domen Kožar (iElectric) Date: 2010-06-25 13:57
I agree — not the best example, here is a better one explaining what behavior should not exist:

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('foobar', action='store')
>>> parser.add_argument('foobar2', nargs='?')
>>> parser.add_argument('foobar3', nargs='*')
>>> print parser.parse_args(['foo', '--', 'foo3', 'foo3'])
Namespace(foobar='foo', foobar2='foo3', foobar3=['foo3'])


I would expect both foo3 to be part of foobar3. This does not happen because of foobar2 argument eating zero or one argument. Arguments after -- should be left unparsed as such behavior was in optparse.
msg108626 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-06-25 20:26
Still looks like it is working as designed to me.  Argpase is *parsing* the arguments, that's part of the whole point.  What you are observing is what I would expect from a well behaved unix command (*arguments* before the -- are just arguments, only the interpretation of options is stopped by --).
msg108652 - (view) Author: Domen Kožar (iElectric) Date: 2010-06-25 23:47
Optparse behaved like that, how would one get the same results with argparse? That is by having variable positional parameters to command. And at the same time stop at --.
msg108676 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-06-26 01:35
Optparse left you to parse the arguments:

>>> import optparse
>>> p = optparse.OptionParser()
>>> p.add_option('--test', action="store_true")
<Option at 0x96521bc: --test>
>>> p.parse_args(['--test', 'foo', '--', 'foo2'])
(<Values at 0x97020a4: {'test': True}>, ['foo', 'foo2'])

As you can see, the 'foo' before the '--' is one of the returned arguments.  So argparse is behaving exactly the same way as optparse in this instance, except that, as I said, it is *parsing* the arguments.
msg108842 - (view) Author: Domen Kožar (iElectric) Date: 2010-06-28 17:04
I always used optparse like this, using the rargs (maybe this is not main intention, but worked so far):

>>> import optparse
>>> parser = optparse.OptionParser()
>>> parser.add_option('--test', action='store_true')
<Option at 0x7f55bdb32dd0: --test>
>>> parser.parse_args(['--test', 'foo', 'foo1', '--', 'foo2', 'foo3'])
(<Values at 0x7f55bda89170: {'test': True}>, ['foo', 'foo1', 'foo2', 'foo3'])
>>> parser.rargs
['foo2', 'foo3']
>>> parser.largs
['foo', 'foo1']
msg108894 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-06-29 07:12
No, this is not intentional.  Look at the docs for these attributes:

parser.largs
    the current list of leftover arguments, ie. arguments that have been consumed but are neither options nor option arguments. Feel free to modify parser.largs, e.g. by adding more arguments to it. (This list will become args, the second return value of parse_args().)
parser.rargs
    the current list of remaining arguments, ie. with opt_str and value (if applicable) removed, and only the arguments following them still there. Feel free to modify parser.rargs, e.g. by consuming more arguments.


In short, they are used *while* parsing options, and have no meaning afterwards; that the OptionParser stops moving items from rargs to largs after the "--" is an implementation detail.
History
Date User Action Args
2010-06-29 07:12:52georg.brandlsetmessages: + msg108894
2010-06-28 17:04:52iElectricsetmessages: + msg108842
2010-06-26 01:35:24r.david.murraysetmessages: + msg108676
2010-06-25 23:47:42iElectricsetmessages: + msg108652
2010-06-25 20:26:20r.david.murraysetnosy: + r.david.murray
messages: + msg108626
2010-06-25 13:57:11iElectricsetmessages: + msg108598
2010-06-25 12:25:16georg.brandlsetstatus: open -> closed

nosy: + georg.brandl
messages: + msg108591

resolution: invalid
2010-06-25 01:40:14eric.smithsetnosy: + eric.smith
2010-06-25 01:33:07benjamin.petersonsetassignee: bethard

nosy: + bethard
2010-06-24 23:34:57iElectriccreate