This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: argparse: nargs could accept range of options count
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: Danh, alclarks, atfrase, bethard, brandtbucher, calestyo, paul.j3, rhettinger, shihai1991, uniqueg, wm
Priority: normal Keywords: patch

Created on 2011-02-28 16:51 by wm, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
argparse-nargs.patch wm, 2011-02-28 16:51 patch
test.py wm, 2011-03-02 20:24
issue11354.patch wm, 2011-04-11 17:40 patch (lib & tests) review
prelimary.patch paul.j3, 2013-05-08 22:32 review
nargsrange.patch paul.j3, 2013-05-09 23:29 review
Messages (22)
msg129714 - (view) Author: Wojciech Muła (wm) Date: 2011-02-28 16:51
Hi, sometimes it is needed to grab variable, but limited, number of options.
For example --point could accept 2, 3 or 4 values: (x,y) or (x,y,z) or
(x,y,z,w). Current version of argparse requires postprocessing:

	parser.add_argument('--point', action='append', default=[])
	nmps = parser.parse_args()
	if not (2 <= len(nmsp.point) <= 4):
		raise argparse.ArgumentTypeError("--point expects 2, 3 or 4 values")

I propose to allow pass range of options count to nargs, including
lower/upper bound. For example:
	
	parser.add_argument('--point', nargs=(2,4) )	# from 2 to 4
	parser.add_argument('--foo', nargs=(9, None) )	# at least 9
	parser.add_argument('--bar', nargs=(None, 7) )	# at most 7
	nmsp = parser.parse_args()

I've attached tests and patch made against Python3.2 lib from Debian.

w.
msg129715 - (view) Author: Wojciech Muła (wm) Date: 2011-02-28 16:52
tests
msg129755 - (view) Author: Daniel Haertle (Danh) Date: 2011-03-01 12:57
Hi Wojciech,

in your tests, at

    def test_add_argument10(self):
        "nargs = (0, 1) => optimized to '?'"
        opt = self.add_argument(1, None)
        self.assertEqual(opt.nargs, argparse.ONE_OR_MORE)

you should change "argparse.ONE_OR_MORE" to "argparse.OPTIONAL".
msg129917 - (view) Author: Wojciech Muła (wm) Date: 2011-03-02 20:24
Daniel, thanks for note, fixed.
msg132240 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2011-03-26 14:07
Thanks for the patch. The idea and the approach of the patch look fine. But the patch needs to be against the Python repository:

http://docs.python.org/devguide/patch.html#creating

For the tests, you should integrate your test.py into Lib/test/test_argparse.py.
msg133537 - (view) Author: Wojciech Muła (wm) Date: 2011-04-11 17:40
Steven, thank you for links, I prepared patch against trunk.
All tests passed.
msg162475 - (view) Author: Alex Frase (atfrase) Date: 2012-06-07 16:01
I'm new here so I apologize if this is considered poor etiquette, but I'm just commenting to 'bump' this issue.  My search for a way to accomplish exactly this functionality led me here, and it seems a patch was offered but no action has been taken on it for over a year now.  Alas, it seems I must write a custom Action handler instead.
msg166171 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-07-22 21:17
The tests look like they're testing the right things, but the tests should instead be written like the rest of the argparse tests. For example, look at TestOptionalsNargs3 and TestPositionalsNargs2. You could write your tests to look something like those, e.g.

class TestOptionalsNargs1_3(ParserTestCase):

    argument_signatures = [Sig('-x', nargs=(1, 3))]
    failures = ['a', '-x', '-x a b c d']
    successes = [
        ('', NS(x=None)),
        ('-x a', NS(x=['a'])),
        ('-x a b', NS(x=['a', 'b'])),
        ('-x a b c', NS(x=['a', 'b', 'c'])),
    ]

Also, a complete patch will need to document the new feature in the Python documentation, among other things. See the details described here:

http://docs.python.org/devguide/patch.html#preparation
msg188743 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-05-08 22:32
Wouldn't it be simpler to use the re {m,n} notation to grab the appropriate number of arguments?  

In ArgumentParser _get_nargs_pattern we could add:

+        # n to m arguments, nargs is re like {n,m}
+        elif is_mnrep(nargs):
+            nargs_pattern = '([-A]%s)'%nargs

        # all others should be integers

where is_mnrep() tests that the nargs string looks like '{m,n}' (or '{,n}' or '{m,}').  The resulting nargs_pattern will look like

    '([-A]{m,n})'

In _format_args() a similar addition would be:

+        elif is_mnrep(action.nargs):
+            result = '%s%s' % (get_metavar(1)[0], action.nargs)

   'FOO{2,5}'

It would take more code to generate

   FOO FOO [FOO [FOO [FOO]]]

A matching programmer input would be:

   parser.add_argument('Foo', nargs='{2,5}')

though it wouldn't be much work to also translate a tuple.

   parser.add_argument('Foo', nargs=(2,5))

I'll try to test this implementation against wm's tests.
msg188757 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-05-09 08:01
I think this patch should build on http://bugs.python.org/issue9849, which seeks to improve the error checking for nargs.  There, add_argument returns an ArgumentError if the nargs value is not a valid string, interger, or it there is mismatch between a tuple metavar and nargs.  

This range nargs should be tested at the same time, and return a ArgumentError if possible.
msg188795 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-05-09 23:29
This patch adds this `nargs='{m,n}'` or `nargs=(m,n)` feature.

It builds on the `better nargs error message` patch in http://bugs.python.org/msg187754

It includes an argparse.rst paragraph, changes to argparse.py, and additions to test_argparse.py.

The tests include those proposed by wm, rewritten to use the ParserTestCase framework where possible.  I did not give a lot of thought to test names.  The coverage could also use further thought.

As WM noted some range cases are the equivalent to existing nargs options ('{1,}'=='+').  I did not attempt to code or test such equivalences.  Since the '{0,}' form uses regex matching just like '*',
I don't think there is any advantage to making such a translation.  

I convert the tuple version (m,n) to the re string '{m,n}' during the add_argument() testing.  So it is the string form that is added to the parser action.  This is also the format that appears in usage.

The documentation paragraph is:

'{m,n}'. m to n command-line arguments are gathered into a list. This is modeled on the Regular Expression use. '{,n}' gathers up to n arguments. '{m,}' gathers m or more. Thus '{1,}' is the equivalent to '+', and '{,1}' to '?'. A tuple notation is also accepted, '(m,n)', '(None,n)', '(m,None)'. For example:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('--foo', nargs='{2,4}')
    >>> parser.parse_args('--foo a b c'.split())
    Namespace(foo=['a', 'b', 'c'])
    >>> parser.parse_args('--foo a'.split())
    usage: PROG [-h] [--foo FOO{2,4}]
    PROG: error: argument --foo: expected {2,4} arguments
msg204775 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-11-30 04:54
With a minor tweak to `argparse._is_mnrep()`, `nargs='{3}'` would also work.  This means the same as `nargs=3`, so it isn't needed.  But it is consistent with the idea of accepting Regular Expression syntax where it makes sense.  `nargs='{2,3}?'` also works, though I think that's just the same as `nargs=2`.
msg355901 - (view) Author: Alex (alclarks) * Date: 2019-11-03 13:46
Hi, I'm a new contributor looking for issues to work on. This looks like a nice enhancement, would it be a good idea for me to update the patch and raise a pull request?
msg355902 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2019-11-03 15:15
I think the answer is 'yes'(It's over 8 years~).
And tons of argparse'work need to be finished;)
msg355904 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2019-11-03 16:40
A couple of quick observations:

- this is one of the first patches that I worked on, so the details aren't fresh in my mind.  A glance at my latest patch show that this isn't a trivial change.  

- nargs changes affect the input handling, the parsing, help and usage formatting, and error formatting.  As a result, nargs are referenced in several places in the code.  That complicates maintenance and the addition of new features.  Help formatting is particularly brittle; just look at the number of issues that deal with 'metavar' to get a sense of that.

- At one point I tried to refactor the code to consolidate the nargs handling in a few functions.  I don't recall if I posted that as a patch.

- The first posts on this topic suggested a (n,m) notation; I proposed a regex like {n,m}.  There wasn't any further discussion.

- Note that the initial posts were in 2011 when Steven (the original author) was involved.  I posted in 2013.  There hasn't been any further action until now.  I don't recall much interest in this topic on Stackoverflow either.  So my sense is that this isn't a pressing issue.

- It's easy to test for this range after parsing, with '*' or '+' nargs.  So the main thing this patch adds is in the help/usage display.  It doesn't add significant functionality.

- cross links:

https://bugs.python.org/issue9849 - Argparse needs better error handling for nargs

https://bugs.python.org/issue16468 - argparse only supports iterable choices (recently closed)
msg355959 - (view) Author: Alex (alclarks) * Date: 2019-11-04 16:24
I've had a look at the most recent patch and the code surrounding it, and I would be happy to take on the work to update the code and testing.

However,
> - It's easy to test for this range after parsing, with '*' or '+' nargs.  So the main thing this patch adds is in the help/usage display.  It doesn't add significant functionality.

I didn't initially consider this.

I'd still be happy to finish this off, but if the general feeling is that contribution time would be better spent elsewhere then that's also fine.
msg356558 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-11-13 21:37
Do we have examples of real world APIs that actually need this functionality? Offhand, I don't recall having worked with any command-line tool that would accept "at least two but no more than four filenames" for example.

Given that this isn't trivial to implement, we should take a moment to evaluate whether users actually need it.  We had eight years since the original proposal without anyone chiming in to say, "yes, I need this".
msg356587 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2019-11-14 09:16
Could we close some bpo with a long history? If some user need this feature in future we could reopen it.
msg356687 - (view) Author: Alex (alclarks) * Date: 2019-11-15 16:34
Weighing up how little demand there seems to be for this, and how easy it is to achieve the same effect with post-processing within a hypothetical script that happens to need it - I agree with closing it.
msg373435 - (view) Author: Christoph Anton Mitterer (calestyo) Date: 2020-07-10 00:20
Next to code readability, there's IMO one could reason to properly support this would be a clean and easy way to get proper help strings for such options.

Of course I can do something like:
parser = argparse.ArgumentParser()
parser.add_argument("--foo", nargs="+", help="Mae govannen", metavar=("bar", "baz"))
args = parser.parse_args()

and later check that, say, only 2 arguments are allowed.

But the help text will be an ugly:
>$ python3 f.py --help
>usage: f.py [-h] [--foo bar [baz ...]]
>
>optional arguments:
>  -h, --help           show this help message and exit
>  --foo bar [baz ...]  Mae govannen

indicating that >1 options were allowed.
msg389162 - (view) Author: Alex Kanitz (uniqueg) Date: 2021-03-20 14:58
Given that people were asking for real-world use cases, here's one: high-throughput sequencing, e.g., in RNA-Seq (https://en.wikipedia.org/wiki/RNA-Seq), typically yields either one or two output files, depending on the type of the sequencing library. As the processing of these library types is very similar, bioinformatics tools dealing with these inputs thus typically have APIs that take exactly 1 or 2 files as inputs. As there is quite a wide range of tools for dealing with such inputs, several of which implemented in Python, I believe there is indeed an interest in this functionality. On a more conceptual note: it is also consistent with similar and often-used functionality in regexes, from which, I suppose, the '+' and '*' notation is borrowed.

Currently implementing such a tool, I ran into the argparse limitation described here: either I (a) use a positional param with nargs=1 for the first file and define an optional param for the second file (inconsistent, non-intuitive and semantically incorrect API, because if there IS a second file, it is not really optional), (b) use nargs='+', do the admittedly simple post-processing/validation and either ignore keep the auto-generated usage string(wrong/misleading), hardcode the correct usage string (maintenance burden because of several optional params) or apply this patch (or just the auto-usage generation function), which seems rather expensive, or (c) have the user pass the second file in one string, separated by a comma or similar (also not very intuitive and needs some checks to ensure that the filename/s don't actually include commas).
msg391843 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2021-04-25 07:05
Post parsing testing for the correct number of strings is the easy part.

It's the auto-generate usage that's harder to do right, especially if we wanted to enable the metavar tuple option.  Clean usage for '+' is messy enough.
History
Date User Action Args
2022-04-11 14:57:13adminsetgithub: 55563
2021-04-25 07:05:49paul.j3setmessages: + msg391843
2021-03-20 14:58:58uniquegsetnosy: + uniqueg
messages: + msg389162
2020-07-10 00:20:24calestyosetnosy: + calestyo
messages: + msg373435
2019-11-15 17:20:18rhettingersetstatus: open -> closed
resolution: rejected
stage: patch review -> resolved
2019-11-15 16:34:20alclarkssetmessages: + msg356687
2019-11-14 09:16:09shihai1991setmessages: + msg356587
2019-11-13 21:37:19rhettingersetmessages: + msg356558
2019-11-13 20:49:16brandtbuchersetnosy: + brandtbucher
2019-11-04 16:24:46alclarkssetmessages: + msg355959
2019-11-03 16:40:24paul.j3setnosy: + rhettinger
messages: + msg355904
2019-11-03 15:15:06shihai1991setnosy: + shihai1991
messages: + msg355902
2019-11-03 13:46:27alclarkssetnosy: + alclarks
messages: + msg355901
2013-11-30 04:54:14paul.j3setmessages: + msg204775
2013-05-09 23:29:08paul.j3setfiles: + nargsrange.patch

messages: + msg188795
2013-05-09 08:01:33paul.j3setmessages: + msg188757
2013-05-08 22:32:57paul.j3setfiles: + prelimary.patch
nosy: + paul.j3
messages: + msg188743

2012-07-22 21:17:50bethardsetmessages: + msg166171
versions: + Python 3.4, - Python 3.3
2012-06-07 16:01:57atfrasesetnosy: + atfrase
messages: + msg162475
2011-04-11 17:40:23wmsetfiles: + issue11354.patch

messages: + msg133537
2011-03-26 14:07:16bethardsetstage: patch review
messages: + msg132240
versions: - Python 3.2
2011-03-02 21:46:34SilentGhostsetnosy: + bethard
2011-03-02 20:24:25wmsetfiles: + test.py

messages: + msg129917
2011-03-02 20:22:32wmsetfiles: - test.py
2011-03-01 12:57:50Danhsetnosy: + Danh
messages: + msg129755
2011-02-28 16:52:26wmsetfiles: + test.py

messages: + msg129715
2011-02-28 16:51:33wmcreate