Title: argparse: Problem with defaults for variable nargs when using choices
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.6, Python 3.5, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: bethard Nosy List: Sworddragon, berker.peksag, bethard, ced, eric.araujo, eric.smith, paul.j3, regis, thesociable
Priority: high Keywords: patch

Created on 2010-08-17 09:19 by thesociable, last changed 2017-01-20 01:28 by paul.j3.

File name Uploaded Description Edit
issue9625.diff bethard, 2012-07-21 22:19 review
issue9625.patch ced, 2012-11-03 16:00 review
issue9625_1.patch paul.j3, 2013-06-27 06:33 review
issue9625_2.patch paul.j3, 2013-07-03 21:36 review
notes.txt paul.j3, 2014-05-02 02:04
Messages (12)
msg114108 - (view) Author: Martin Pengelly-Phillips (thesociable) Date: 2010-08-17 09:19
Variable argument count plays badly with choices.

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('choices', nargs='*', default='a', choices=['a',
'b', 'c'])
>>> args = parser.parse_args()
>>> print type(args.choices)
<type 'str'>
>>> args = parser.parse_args(['a'])
>>> print type(args.choices)
<type 'list'>

If the user specifies the value on the command line then a list is used, but if the value comes from the default a string is used.
Unfortunately, changing default to a list value gives an error:
error: argument choices: invalid choice: ['a'] (choose from 'a', 'b',

Additionally, this means it is also not possible to use default=['a', 'c']. 

The current workaround is to create a custom type:

def my_type(string):
    if string not in ['a', 'b', 'c']:
        raise TypeError
    return string
msg151780 - (view) Author: Michał M. (regis) Date: 2012-01-22 17:55
Maybe it will sound strange, but what is this task REALLY about? I mean - I can see two problems here, but no clear information about which problem is a real problem and - if it is - what is the expected behavior.

Problems I can see are:
1) Type of returned value changes depending on the value source (list for user provided or list for default)
2) It's impossible to set list as 'default'

I understand that this task concentrates on 1st one, however changing behavior described in 2nd would - in my oppinion - fix 1st one too - am I right? User would "define" the returned type by defining the 'default' as a list or string. 

But - if we assume that we only discuss 1st problem here - what is the expected behavior? Provided workaround is suggesting that we expect string, but - basing on current nargs behavior and "intuition" - I'd rather expect to get a list, when I define nargs with "*" or "+". This sounds "natural" for me.

Could someone explain me the problem and the expected behavior in a clear way?

Sorry if I'm not clear, but it's my first post here and maybe I've missed something in the description or I assume something that is not true (especially that the task is quite old... ;) ).
msg151807 - (view) Author: Michał M. (regis) Date: 2012-01-23 10:50
Of course I've made a mistake:

"list for user provided or list for default"

should be:

"list for user provided or STRING for default"
msg151857 - (view) Author: Martin Pengelly-Phillips (thesociable) Date: 2012-01-23 22:23
The real issue is that the choices flag does not work with a default flag and * nargs.

The following works as expected:
>>> parser.add_argument('chosen', nargs='*', default=['a'])
>>> print(parser.parse_args())
>>> print(parser.parse_args(['a', 'b']))
Namespace(chosen=['a', 'b'])

Introducing a choices constraint breaks down when using the defaults:
>>> parser.add_argument('chosen', nargs='*', default=['a'], choices=['a', 'b'])
>>> print(parser.parse_args(['a']))
>>> print(parser.parse_args())
error: argument chosen: invalid choice: ['a'] (choose from 'a', 'b')

I would expect instead to have Namespace.chosen populated with the default list as before, but the choices constraint check does not validate correctly.

I think that changing the choices constraint logic to iterate over the default values if nargs results in a list would be a possible solution.
msg166086 - (view) Author: Steven Bethard (bethard) * (Python committer) Date: 2012-07-21 22:19
I agree that this looks like a bug. I think the fix is something like the attached patch, but it needs some tests to make sure that it fixes your problem.
msg174642 - (view) Author: Cédric Krier (ced) * Date: 2012-11-03 16:00
Here is a new version of the patch with tests
msg191224 - (view) Author: Cédric Krier (ced) * Date: 2013-06-15 19:02
msg191932 - (view) Author: paul j3 (paul.j3) * Date: 2013-06-27 06:33
I've added 2 more tests, 

one with default='c', which worked before.

one with default=['a','b'], which only works with this change. is useful reference, since it documents
the differences between nargs="?" and nargs="*", and their handling of
their defaults.
msg192258 - (view) Author: paul j3 (paul.j3) * Date: 2013-07-03 21:36
Change "choices='abc'" to "choices=['a', 'b', 'c']", as discussed in issue 16977  (use of string choices is a bad example)
msg192716 - (view) Author: paul j3 (paul.j3) * Date: 2013-07-09 03:07
The patch I just posted to uses this fix.
msg217677 - (view) Author: paul j3 (paul.j3) * Date: 2014-05-01 04:46
There's a complicating issue - should these default values be passed through the type function?

In most cases in `_get_values`, the string first goes through `_get_value`, and then to `_check_value`.

For example the 'else:' case:

            value = [self._get_value(action, v) for v in arg_strings]
            for v in value:
                self._check_value(action, v)

The '*' positional case could coded the same way, allowing:


and objecting to 

         default=[6,'7','a'] # out of range string or int or invalid value

This does impose a further constraint on the 'type' function, that it accepts a converted value.  e.g. int(1) is as valid as int('1').

But we need to be careful that this case is handled in a way that is consistent with other defaults (including the recent change that delayed evaluating defaults till the end).
msg285865 - (view) Author: paul j3 (paul.j3) * Date: 2017-01-20 01:28
Recent StackOverFlow question related to this issue
Date User Action Args
2017-01-20 01:28:04paul.j3setmessages: + msg285865
2017-01-19 23:02:30paul.j3setpriority: normal -> high
2016-06-12 14:30:54berker.peksagsetnosy: + berker.peksag
stage: needs patch -> patch review

versions: + Python 3.5, Python 3.6, - Python 3.2
2016-06-12 14:30:29berker.peksaglinkissue27227 superseder
2015-08-29 10:19:14Sworddragonsetnosy: + Sworddragon
2014-05-02 02:04:07paul.j3setfiles: + notes.txt
2014-05-01 04:46:43paul.j3setmessages: + msg217677
2013-07-09 03:07:58paul.j3setmessages: + msg192716
2013-07-03 21:36:35paul.j3setfiles: + issue9625_2.patch

messages: + msg192258
2013-06-27 06:33:20paul.j3setfiles: + issue9625_1.patch

messages: + msg191932
2013-06-26 20:36:39paul.j3setnosy: + paul.j3
2013-06-15 19:02:34cedsetmessages: + msg191224
2012-11-03 16:00:08cedsetfiles: + issue9625.patch
nosy: + ced
messages: + msg174642

2012-07-21 22:19:22bethardsetfiles: + issue9625.diff
keywords: + patch
messages: + msg166086
2012-01-23 22:23:14thesociablesetmessages: + msg151857
title: argparse: Problem with defaults for variable nargs -> argparse: Problem with defaults for variable nargs when using choices
2012-01-23 10:50:51regissetmessages: + msg151807
2012-01-22 17:55:46regissetnosy: + regis
messages: + msg151780
2010-11-02 22:01:00eric.araujosetnosy: + eric.araujo
2010-08-21 22:39:06georg.brandlsetassignee: bethard
2010-08-17 19:08:29eric.smithsetnosy: + eric.smith
2010-08-17 13:22:56bethardsetnosy: + bethard
stage: needs patch

versions: + Python 2.7, Python 3.2, - Python 2.6
2010-08-17 09:19:03thesociablecreate