classification
Title: Improve error messages for argparse choices using enum
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: miss-islington, paul.j3, rhettinger, shangxiao, xtreak
Priority: normal Keywords: patch

Created on 2020-11-29 10:20 by shangxiao, last changed 2020-11-30 21:22 by rhettinger. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 23563 merged rhettinger, 2020-11-29 22:33
PR 23573 merged miss-islington, 2020-11-30 17:55
Messages (8)
msg382051 - (view) Author: (shangxiao) * Date: 2020-11-29 10:20
Summary
-------

The argparse module mentions that it will happily accept an enum for the choices argument option [1].  There are currently 2 issues with this:

1. The usage displays non user-friendly values for the enum;
2. The error message for invalid values is not consistent with those provided by iterable-based choices in that it doesn't show the list of available choices. This becomes a problem when you specify the metavar as no choices are displayed at all.


Example of 1
------------

For example using the GameMove example in the argparse documentation:

    class GameMove(Enum):
         ROCK = 'rock'
         PAPER = 'paper'
         SCISSORS = 'scissors'

    parser = argparse.ArgumentParser(prog='game.py')
    parser.add_argument('move', type=GameMove, choices=GameMove)
    parser.print_help()


Gives the usage:

    usage: game.py [-h] {GameMove.ROCK,GameMove.PAPER,GameMove.SCISSORS}

    positional arguments:
      {GameMove.ROCK,GameMove.PAPER,GameMove.SCISSORS}

    optional arguments:
      -h, --help            show this help message and exit


As you can see, the string representations of the valid enum members is used instead of the values.



Example of 2
------------

Below is an example of the error message shown for invalid enum values, with metavar specified:

    parser = argparse.ArgumentParser(prog='game.py')
    parser.add_argument('move', metavar='your-move', type=GameMove, choices=GameMove)
    parser.parse_args(['asdf'])

Gives the following output:

    usage: game.py [-h] your-move
    game.py: error: argument your-move: invalid GameMove value: 'asdf'


This is in contrast with using standard iterable-based choices:

    parser = argparse.ArgumentParser(prog='game.py')
    parser.add_argument('move', metavar='your-move', choices=[x.value for x in GameMove])
    parser.parse_args(['asdf'])

Gives the following output:

    usage: game.py [-h] your-move
    game.py: error: argument your-move: invalid choice: 'asdf' (choose from 'rock', 'paper', 'scissors')


Analysis of 2
--------------

The reason for this behaviour is because argparse attempts to convert to the correct type before checking the choices membership and an invalid enum value results in the first error message.

It would be helpful (& consistent) if the choices are also displayed along with the invalid value message.


Possible Solution
-----------------

I believe that both issues could be solved by using the enum member values in the choices and then checking for choice membership _before_ the value is converted to an enum (ie for enums only).



[1] https://docs.python.org/3/library/argparse.html#choices
msg382054 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-11-29 11:56
I'm inclined to state in the docs that choices is designed to work with collections of strings and that there is no special support for enums.

Paul, what do you think?
msg382058 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2020-11-29 12:14
https://bugs.python.org/issue25061 also had some discussion over error message display for enums
msg382063 - (view) Author: (shangxiao) * Date: 2020-11-29 13:39
Oh apologies, I had "open" selected when I searched for prior issues.

Upon reading that issue I agree with Mr Hettinger's points about enum values not being a concern of the parser.

The solution for my needs was simple enough: I made my own action which simply set choices based on member values (I'm using user-friendly strings) and converted to the correct type upon being called [1].

Instead of removing any mention of enums from the docs - would a small example showing how to deal with them be worthwhile?


[1]
def enum_action_factory(enum_class):

    class EnumAction(argparse.Action):
        def __init__(self, option_strings, dest, **kwargs):
            kwargs["choices"] = [member.value for member in enum_class]
            super().__init__(option_strings, dest, **kwargs)

        def __call__(self, parser, namespace, values, option_string):
            if isinstance(values, str):
                converted_values = enum_class(values)
            else:
                converted_values = [enum_class(value) for value in values]
            setattr(namespace, self.dest, converted_values)

    return EnumAction
msg382094 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2020-11-29 21:22
choices is fine for a few strings, but quickly becomes awkward with other types and large numbers.  The testing isn't an issue, since it just does a simple `in/contains` test.  But display, whether in usage, help or error, is problematic if you try anything too fancy.  metavar gets around some of those issues, but doesn't change the error messages.

Custom type or action is the best alternative.

I'm in favor omitting the enums mention in the docs, since it seems to be more confusing than helpful.
msg382096 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-11-29 22:41
How does PR 23563 look to you all?
msg382168 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-11-30 17:55
New changeset 7f82f22eba1312617e1aa19cb916da1aae1609a4 by Raymond Hettinger in branch 'master':
bpo-42501:  Revise the usage note for Enums with the choices (GH-23563)
https://github.com/python/cpython/commit/7f82f22eba1312617e1aa19cb916da1aae1609a4
msg382193 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-11-30 21:21
New changeset aab93903347ec6d7f23dda2994dd26f6111d19d2 by Miss Islington (bot) in branch '3.9':
bpo-42501:  Revise the usage note for Enums with the choices (GH-23563) (GH-23573)
https://github.com/python/cpython/commit/aab93903347ec6d7f23dda2994dd26f6111d19d2
History
Date User Action Args
2020-11-30 21:22:19rhettingersetstatus: open -> closed
resolution: fixed
components: + Documentation, - Library (Lib)
stage: patch review -> resolved
2020-11-30 21:21:22rhettingersetmessages: + msg382193
2020-11-30 17:55:40rhettingersetmessages: + msg382168
2020-11-30 17:55:29miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request22454
2020-11-29 22:54:37rhettingersetassignee: rhettinger
2020-11-29 22:41:02rhettingersetmessages: + msg382096
2020-11-29 22:33:25rhettingersetkeywords: + patch
stage: patch review
pull_requests: + pull_request22444
2020-11-29 21:22:39paul.j3setmessages: + msg382094
2020-11-29 13:39:12shangxiaosetmessages: + msg382063
2020-11-29 12:14:11xtreaksetnosy: + xtreak
messages: + msg382058
2020-11-29 11:56:17rhettingersetmessages: + msg382054
2020-11-29 11:14:24xtreaksetnosy: + rhettinger, paul.j3
2020-11-29 10:20:31shangxiaocreate