classification
Title: argparse optionals with nargs='+' can't be followed by positionals
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.3, Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Kotan, bethard, catherine, elsdoerfer, eric.araujo, wrobell
Priority: normal Keywords: patch

Created on 2010-07-23 10:46 by bethard, last changed 2011-08-05 15:20 by eric.araujo.

Files
File name Uploaded Description Edit
test_pos_after_var_args.patch catherine, 2010-08-02 18:42 review
issue9338_argparse.patch Kotan, 2010-11-20 19:05 review
Messages (9)
msg111270 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-07-23 10:46
[From the old argparse tracker: http://code.google.com/p/argparse/issues/detail?id=20]

You can't follow a nargs='+' optional argument with a positional argument:

>>> import argparse
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--badger', nargs='+')
>>> parser.add_argument('spam')
>>> parser.parse_args('--badger A B C D'.split())
usage: PROG [-h] [--badger BADGER [BADGER ...]] spam
PROG: error: too few arguments

Ideally, this should produce:

>>> parser.parse_args('--badger A B C D'.split())
Namespace(badger=['A', 'B', 'C'], spam='D')

The problem is that the nargs='+' causes the optional to consume all the arguments following it, even though we should know that we need to save one for the final positional argument.

A workaround is to specify '--', e.g.:

>>> parser.parse_args('--badger A B C -- D'.split())
Namespace(badger=['A', 'B', 'C'], spam='D')

The problem arises from the fact that argparse uses regular-expression style matching for positional arguments, but it does that separately from what it does for optional arguments.

One solution might be to build a regular expression of the possible things a parser could match. So given a parser like::

  parser = argparse.ArgumentParser()
  parser.add_argument('-w')
  parser.add_argument('-x', nargs='+')
  parser.add_argument('y')
  parser.add_argument('z', nargs='*')

the regular expression might look something like (where positionals have been replaced by the character A)::

  (-w A)? (-x A+)? A (-w A)? (-x A+)? A* (-w A)? (-x A+)?

Note that the optionals can appear between any positionals, so I have to repeat their regular expressions multiple times. Because of this, I worry about how big the regular expression might grow to be for large parsers. But maybe this is the right way to solve the problem.
msg111280 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2010-07-23 11:26
This is definitely a different bug from the one you just marked it as a duplicate of.
msg111282 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-07-23 11:41
I read too fast, I’m sorry for that.
msg112511 - (view) Author: Catherine Devlin (catherine) Date: 2010-08-02 18:42
Here's a unit test for the simplest cases.
msg121013 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-12 02:32
Looks good to me.  Do you want to propose a code patch too, and/or more tests for non-simple cases?
msg121720 - (view) Author: Daniel Albeseder (Kotan) Date: 2010-11-20 19:05
My attached patch adds the "--" between the optionals and the arguments, if there are optionals which have variable length and at least some positional argument can be provided.

Patch is for python 3.2 svn revision 86553.
msg122016 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-21 23:20
Patch looks good, thanks!  Does it integrate Catherine’s tests?
msg141639 - (view) Author: wrobell (wrobell) Date: 2011-08-04 23:03
is there a chance to fix this issue?
msg141660 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-05 15:20
There is.  Someone wanting to help could reply to the question I asked :)
History
Date User Action Args
2011-08-05 15:20:14eric.araujosettype: enhancement -> behavior
messages: + msg141660
versions: + Python 3.3
2011-08-04 23:03:52wrobellsetnosy: + wrobell
messages: + msg141639
2010-11-21 23:20:04eric.araujosetnosy: bethard, eric.araujo, catherine, elsdoerfer, Kotan
messages: + msg122016
components: + Library (Lib)
stage: needs patch -> patch review
2010-11-20 19:05:01Kotansetfiles: + issue9338_argparse.patch
nosy: + Kotan
messages: + msg121720

2010-11-12 02:32:40eric.araujosetmessages: + msg121013
versions: + Python 2.7, Python 3.2
2010-08-12 10:17:53elsdoerfersetnosy: + elsdoerfer
2010-08-02 18:42:03catherinesetfiles: + test_pos_after_var_args.patch

nosy: + catherine
messages: + msg112511

keywords: + patch
2010-07-23 11:41:47eric.araujosetnosy: + eric.araujo
messages: + msg111282
2010-07-23 11:26:16bethardsetstatus: closed -> open
resolution: duplicate ->
messages: + msg111280

superseder: argparse does not accept options taking arguments beginning with dash (regression from optparse) ->
stage: committed/rejected -> needs patch
2010-07-23 10:47:34eric.araujosetstatus: open -> closed
resolution: duplicate
superseder: argparse does not accept options taking arguments beginning with dash (regression from optparse)
stage: needs patch -> committed/rejected
2010-07-23 10:46:27bethardcreate