Title: argparse: misbehavior when combining positionals and choices
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: Nosy List: jameshcorbett, paul.j3, rhettinger
Priority: normal Keywords: patch

Created on 2020-06-20 02:15 by jameshcorbett, last changed 2020-06-21 03:22 by paul.j3. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 20997 closed jameshcorbett, 2020-06-20 02:29
Messages (3)
msg371915 - (view) Author: James Corbett (jameshcorbett) * Date: 2020-06-20 02:15
The `argparse.ArgumentParser` sometimes rejects positional arguments with no arguments when `choices` is set and `nargs="*"`. 

When there are no arguments and `nargs` is `"*"`, the default value is chosen, or `[]` if there is no default value. This value is then checked against `choices` and an error message is printed if the value is not in `choices`. However, sometimes the value is intentionally not in `choices`, and this leads to problems. An example will explain this much better, and show that the issue only occurs with the particular combination of positionals, `nargs="*"`, and `choices`:

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument("foo", choices=["a", "b", "c"], nargs="*")
>>> parser.add_argument("--bar", choices=["d", "e", "f"], nargs="*")
>>> parser.add_argument('--baz', type=int, choices=range(5, 10), default="20")
>>> parser.parse_args("a --bar".split())
Namespace(foo=['a'], bar=[], baz=20)
>>> parser.parse_args(["a"])
Namespace(foo=['a'], bar=None, baz=20)
>>> parser.parse_args([])
usage: [-h] [--bar [{d,e,f} ...]] [--baz {5,6,7,8,9}] [{a,b,c} ...]
: error: argument foo: invalid choice: [] (choose from 'a', 'b', 'c')

In this case I could have got around the last error by adding `[]` to choices, but that pollutes the help and usage messages.
msg371980 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2020-06-21 03:17
The error is raised in ._get_values with this section:

        # when nargs='*' on a positional, if there were no command-line
        # args, use the default if it is anything other than None
        elif (not arg_strings and action.nargs == ZERO_OR_MORE and
              not action.option_strings):
            if action.default is not None:
                value = action.default
                value = arg_strings
            self._check_value(action, value)

An empty 'arg_strings' satisfies the nargs='*', so 'value' is set to that, and passed to _check_value.

We could avoid this by setting the default to one of the valid choices, e.g. 'a'.  But then the attribute would be that string, not a list with that string.

I suspect this issue has come up before, since I don't this code has been changed in a long time.
msg371981 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2020-06-21 03:22
It is an old unresolved issue (which resurfaced a few months ago)
argparse: Problem with defaults for variable nargs when using choices
Date User Action Args
2020-06-21 03:22:38paul.j3setstatus: open -> closed
stage: patch review -> resolved
2020-06-21 03:22:12paul.j3setresolution: duplicate
messages: + msg371981
2020-06-21 03:17:54paul.j3setmessages: + msg371980
2020-06-20 02:29:09jameshcorbettsetkeywords: + patch
stage: patch review
pull_requests: + pull_request20172
2020-06-20 02:23:45xtreaksetnosy: + rhettinger, paul.j3
2020-06-20 02:15:23jameshcorbettcreate