This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: argparse: Problem with defaults for variable nargs when using choices
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jan Hutař, Sworddragon, berker.peksag, bethard, ced, eric.araujo, eric.smith, jameshcorbett, lanzz, macfreek, paul.j3, regis, rhettinger, sebix, thesociable, zumoshi
Priority: normal Keywords: patch

Created on 2010-08-17 09:19 by thesociable, last changed 2022-04-11 14:57 by admin.

File name Uploaded Description Edit
issue9625.diff bethard, 2012-07-21 22:19 review
issue9625.patch ced, 2012-11-03 16:00 review
issue9625_1.patch paul.j3, 2013-06-27 06:33 review
issue9625_2.patch paul.j3, 2013-07-03 21:36 review
notes.txt paul.j3, 2014-05-02 02:04
Messages (17)
msg114108 - (view) Author: Martin Pengelly-Phillips (thesociable) Date: 2010-08-17 09:19
Variable argument count plays badly with choices.

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('choices', nargs='*', default='a', choices=['a',
'b', 'c'])
>>> args = parser.parse_args()
>>> print type(args.choices)
<type 'str'>
>>> args = parser.parse_args(['a'])
>>> print type(args.choices)
<type 'list'>

If the user specifies the value on the command line then a list is used, but if the value comes from the default a string is used.
Unfortunately, changing default to a list value gives an error:
error: argument choices: invalid choice: ['a'] (choose from 'a', 'b',

Additionally, this means it is also not possible to use default=['a', 'c']. 

The current workaround is to create a custom type:

def my_type(string):
    if string not in ['a', 'b', 'c']:
        raise TypeError
    return string
msg151780 - (view) Author: Michał M. (regis) Date: 2012-01-22 17:55
Maybe it will sound strange, but what is this task REALLY about? I mean - I can see two problems here, but no clear information about which problem is a real problem and - if it is - what is the expected behavior.

Problems I can see are:
1) Type of returned value changes depending on the value source (list for user provided or list for default)
2) It's impossible to set list as 'default'

I understand that this task concentrates on 1st one, however changing behavior described in 2nd would - in my oppinion - fix 1st one too - am I right? User would "define" the returned type by defining the 'default' as a list or string. 

But - if we assume that we only discuss 1st problem here - what is the expected behavior? Provided workaround is suggesting that we expect string, but - basing on current nargs behavior and "intuition" - I'd rather expect to get a list, when I define nargs with "*" or "+". This sounds "natural" for me.

Could someone explain me the problem and the expected behavior in a clear way?

Sorry if I'm not clear, but it's my first post here and maybe I've missed something in the description or I assume something that is not true (especially that the task is quite old... ;) ).
msg151807 - (view) Author: Michał M. (regis) Date: 2012-01-23 10:50
Of course I've made a mistake:

"list for user provided or list for default"

should be:

"list for user provided or STRING for default"
msg151857 - (view) Author: Martin Pengelly-Phillips (thesociable) Date: 2012-01-23 22:23
The real issue is that the choices flag does not work with a default flag and * nargs.

The following works as expected:
>>> parser.add_argument('chosen', nargs='*', default=['a'])
>>> print(parser.parse_args())
>>> print(parser.parse_args(['a', 'b']))
Namespace(chosen=['a', 'b'])

Introducing a choices constraint breaks down when using the defaults:
>>> parser.add_argument('chosen', nargs='*', default=['a'], choices=['a', 'b'])
>>> print(parser.parse_args(['a']))
>>> print(parser.parse_args())
error: argument chosen: invalid choice: ['a'] (choose from 'a', 'b')

I would expect instead to have Namespace.chosen populated with the default list as before, but the choices constraint check does not validate correctly.

I think that changing the choices constraint logic to iterate over the default values if nargs results in a list would be a possible solution.
msg166086 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-07-21 22:19
I agree that this looks like a bug. I think the fix is something like the attached patch, but it needs some tests to make sure that it fixes your problem.
msg174642 - (view) Author: Cédric Krier (ced) * Date: 2012-11-03 16:00
Here is a new version of the patch with tests
msg191224 - (view) Author: Cédric Krier (ced) * Date: 2013-06-15 19:02
msg191932 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-06-27 06:33
I've added 2 more tests, 

one with default='c', which worked before.

one with default=['a','b'], which only works with this change. is useful reference, since it documents
the differences between nargs="?" and nargs="*", and their handling of
their defaults.
msg192258 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-07-03 21:36
Change "choices='abc'" to "choices=['a', 'b', 'c']", as discussed in issue 16977  (use of string choices is a bad example)
msg192716 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2013-07-09 03:07
The patch I just posted to uses this fix.
msg217677 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2014-05-01 04:46
There's a complicating issue - should these default values be passed through the type function?

In most cases in `_get_values`, the string first goes through `_get_value`, and then to `_check_value`.

For example the 'else:' case:

            value = [self._get_value(action, v) for v in arg_strings]
            for v in value:
                self._check_value(action, v)

The '*' positional case could coded the same way, allowing:


and objecting to 

         default=[6,'7','a'] # out of range string or int or invalid value

This does impose a further constraint on the 'type' function, that it accepts a converted value.  e.g. int(1) is as valid as int('1').

But we need to be careful that this case is handled in a way that is consistent with other defaults (including the recent change that delayed evaluating defaults till the end).
msg285865 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2017-01-20 01:28
Recent StackOverFlow question related to this issue
msg347742 - (view) Author: João Eiras (João Eiras) Date: 2019-07-12 12:55
Another workaround for who might ever need it. The benefit of this solution comparing to a custom type is that argparse will generate the help string properly with the choices, and this solution does workaround the place when the bug happens:

    class Choices(tuple):
        # Python bug
        def __new__(cls, *args, **kwargs):
            x = tuple.__new__(cls, *args, **kwargs)
            Choices.__init__(x, *args, **kwargs)
            return x

        def __init__(self, *args, **kwargs):
            self.default = []

        def __contains__(self, item):
            return tuple.__contains__(self, item) or item is self.default

    choices = Choices(("value1", "value2", "value3", ...))

msg356859 - (view) Author: Mihail Milushev (lanzz) Date: 2019-11-18 11:14
It looks like choices are broken for nargs='*' even without using default:

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('test', nargs='*', choices=['foo', 'bar', 'baz'])
_StoreAction(option_strings=[], dest='test', nargs='*', const=None, default=None, type=None, choices=['foo', 'bar', 'baz'], help=None, metavar=None)
>>> parser.parse_args([])
usage: [-h] [{foo,bar,baz} [{foo,bar,baz} ...]]
: error: argument test: invalid choice: [] (choose from 'foo', 'bar', 'baz')
msg357064 - (view) Author: Jan Hutař (Jan Hutař) Date: 2019-11-20 11:55
I think there is a same issue with "nargs='+'" - if you are aware of the, please ignore me.

$ python3 --version
Python 3.7.5

With "choices=...", there seems to be a problem:

$ cat 
#!/usr/bin/env python3

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('choices', nargs='*',
                    default=['a', 'b', 'c'],
                    choices=['a', 'b', 'c'])
args = parser.parse_args()

$ ./ 
usage: [-h] [{a,b,c} [{a,b,c} ...]] error: argument choices: invalid choice: ['a', 'b', 'c'] (choose from 'a', 'b', 'c')
$ ./ a c
Namespace(choices=['a', 'c'])

but without choices it works:

$ cat 
#!/usr/bin/env python3

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('choices', nargs='*',
                    default=['a', 'b', 'c'])
args = parser.parse_args()

$ ./ 
Namespace(choices=['a', 'b', 'c'])
$ ./ a c
Namespace(choices=['a', 'c'])

As I said, if this is a known issue, I'm sorry for the noise.
msg374581 - (view) Author: James Corbett (jameshcorbett) * Date: 2020-07-29 18:22
I would love to get this issue resolved; it seems like everyone agrees that it's a bug. It came up for me recently: Judging from the comments above, the consensus is that the relevant line, `self._check_value(action, value)` should either be replaced with something like `if isinstance(value, for v in value: self._check_value(action, v)` or be removed entirely.

I think the line should just be removed. I think it's fair to assume that users of `argparse` know what they're doing, so I think they should be allowed to pass default values that conflict with `choices`. Also, removing the line makes the behavior consistent with the optionals, which don't check whether default values are in `choices`. See the below script:

import argparse

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--foo", nargs="+", default=[-1], choices=range(10))
    parser.add_argument("--bar", nargs="*", default=-1, choices=range(10))
    parser.add_argument("pos", nargs="?", default=-1, choices=range(10))
    args = parser.parse_args()

if __name__ == '__main__':

Which yields:
$ python 
Namespace(foo=[-1], bar=-1, pos=-1)
msg374592 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-07-29 23:13
Paul, what is your current thinking on this?
Date User Action Args
2022-04-11 14:57:05adminsetgithub: 53834
2020-12-21 23:36:46rhettingersetassignee: rhettinger ->
2020-12-01 04:52:56zumoshisetnosy: + zumoshi
2020-07-29 23:13:20rhettingersetmessages: + msg374592
2020-07-29 18:22:56jameshcorbettsetnosy: + jameshcorbett
messages: + msg374581
2019-11-20 11:55:54Jan Hutařsetnosy: + Jan Hutař
messages: + msg357064
2019-11-18 11:14:30lanzzsetnosy: + lanzz
messages: + msg356859
2019-08-30 06:26:19rhettingersetpriority: high -> normal
versions: + Python 3.9, - Python 2.7, Python 3.5, Python 3.6
2019-08-30 03:30:47rhettingersetassignee: bethard -> rhettinger

nosy: + rhettinger
2019-07-12 13:00:14João Eirassetnosy: - João Eiras
2019-07-12 12:55:53João Eirassetnosy: + João Eiras
messages: + msg347742
2018-10-16 17:01:43sebixsetnosy: + sebix
2018-04-26 12:05:40macfreeksetnosy: + macfreek
2017-01-20 01:28:04paul.j3setmessages: + msg285865
2017-01-19 23:02:30paul.j3setpriority: normal -> high
2016-06-12 14:30:54berker.peksagsetnosy: + berker.peksag
stage: needs patch -> patch review

versions: + Python 3.5, Python 3.6, - Python 3.2
2016-06-12 14:30:29berker.peksaglinkissue27227 superseder
2015-08-29 10:19:14Sworddragonsetnosy: + Sworddragon
2014-05-02 02:04:07paul.j3setfiles: + notes.txt
2014-05-01 04:46:43paul.j3setmessages: + msg217677
2013-07-09 03:07:58paul.j3setmessages: + msg192716
2013-07-03 21:36:35paul.j3setfiles: + issue9625_2.patch

messages: + msg192258
2013-06-27 06:33:20paul.j3setfiles: + issue9625_1.patch

messages: + msg191932
2013-06-26 20:36:39paul.j3setnosy: + paul.j3
2013-06-15 19:02:34cedsetmessages: + msg191224
2012-11-03 16:00:08cedsetfiles: + issue9625.patch
nosy: + ced
messages: + msg174642

2012-07-21 22:19:22bethardsetfiles: + issue9625.diff
keywords: + patch
messages: + msg166086
2012-01-23 22:23:14thesociablesetmessages: + msg151857
title: argparse: Problem with defaults for variable nargs -> argparse: Problem with defaults for variable nargs when using choices
2012-01-23 10:50:51regissetmessages: + msg151807
2012-01-22 17:55:46regissetnosy: + regis
messages: + msg151780
2010-11-02 22:01:00eric.araujosetnosy: + eric.araujo
2010-08-21 22:39:06georg.brandlsetassignee: bethard
2010-08-17 19:08:29eric.smithsetnosy: + eric.smith
2010-08-17 13:22:56bethardsetnosy: + bethard
stage: needs patch

versions: + Python 2.7, Python 3.2, - Python 2.6
2010-08-17 09:19:03thesociablecreate