Title: Allow dict choices to "transform" values in argpagse
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.8
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: bethard Nosy List: Matthijs Kooijman, bethard, paul.j3, porton, rhettinger, zach.ware
Priority: normal Keywords:

Created on 2018-07-22 12:53 by porton, last changed 2018-09-23 01:24 by paul.j3. This issue is now closed.

Messages (9)
msg322142 - (view) Author: Victor Porton (porton) Date: 2018-07-22 12:53
The below code produces "rock", but it should produce "a". This (to use dict value instead of the key by argparse result) is both to be a new useful feature (for example to map strings in a command line to certain functions or classes provided as dict values) and conform to intuition better.

My feature proposal breaks backward compatibility, but I think no reasonable programmer wrote it in such a way that he relied on the current behavior for `dict` values for `choices`.

import argparse

parser = argparse.ArgumentParser(prog='')
parser.add_argument('move', choices={'rock':'a', 'paper':'b', 'scissors':'c'})
msg322144 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2018-07-22 13:31
I like to think I'm fairly reasonable :).  You can get what you want by specifying `type=choices_dict.get`, or by extracting from the dict manually after calling `parse_args`.
msg322167 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-07-22 20:18
I concur with Zachary.  Downstream processing or transformation of inputs is beyond the scope of argparse which should stick to its core responsibilities of argument extraction.
msg322168 - (view) Author: Victor Porton (porton) Date: 2018-07-22 21:33
"Downstream processing or transformation of inputs is beyond the scope of argparse which should stick to its core responsibilities of argument extraction."

You somehow contradict the official documentation:
"One particularly effective way of handling sub-commands is to combine the use of the add_subparsers() method with calls to set_defaults() so that each subparser knows which Python function it should execute."

The official documentation seems to recommend to pass real Python functions to the parser. So it does do simple "processing or transformation of inputs" which you recommend against.
msg322170 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-07-22 22:18
Victor, please don't try to get your way by adopting a tone of shoving people around or mis-parsing their words try to catch them in a contradiction. There is in fact a boundary between argument parsing logic and application logic. In my view, the proposed behavior crosses that boundary.  Where the docs mention "transformations", they do so in limited ways (such as type conversion) that are tightly bound to argument parsing and are not part of the application logic.

FWIW, I'm not the one you need to convince. Steven Bethard is the module maintainer and arbiter of feature requests.  Recommendations from Zachary and me are meant to help inform his decision.
msg322233 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2018-07-23 16:46
The 'choices' mechanism is a simple 'value in choices' test.  A dictionary (or other mapping) works because it works with 'in'.

'type' as noted is the means that argparse provides for transforming an input string into some other object (most commonly a number with 'int' or 'float').  The choices test is performed after the type transformation.

The 'set_defaults()' with subparsers is offered almost as a parenthetical idea, and has nothing to do with 'choices' or 'type'.  'set_defaults' is just another way of setting default values, and works even with 'dest' which aren't otherwise defined.  If that isn't clear, I'd suggest testing it with the main parser.  

In Python functions are first class objects, and thus can be used just like strings or numbers - assigned to variables, put in lists, etc.  


   adict = {'I':int, 'F':float}
   parser.add_argument('foo', type=lambda x: adict.get(x), choices=adict.values())

the 'type' transforms the commandline string into a the dictionary value, and 'choices' then tests that against the values of the dictionary.  (I had to use `lambda` instead of 'adict.get' directly because of testing that 'type' does.)
msg323673 - (view) Author: Matthijs Kooijman (Matthijs Kooijman) Date: 2018-08-17 21:42
I was today trying to achieve something similar. I also considered the solution using `type` argument, but that does not seem to play well with the `choices` argument. I was going to report that as a bug, but this issue seems similar enough to comment here instead.

The solution proposed by Paul works, in the sense that if 'I' is passed on the commandline, the parsed value because `int` (the actual type, not a string, not sure if Paul really intended that). However, when running --help with his example, you get:

    usage: [-h] {<class 'float'>,<class 'int'>}

  - Since the `choices` argument is used to display help text, `choices` should  contain the values that should be specified on the commandline (e.g. the *inputs* to the `type` converter.
 - Since the *type-converted* value is checked against the `choices` argument, `choices` should contain the *outputs* of the `type` converter.

AFAICS these two constraints cannot be fulfilled at the same time, except when no type conversion happens (or, when converted values can be stringified back to their unconverted value, which works in simple cases, I suppose).

IMHO this looks like a bug: `type` and `choices` do not play well together. Checking specified values against `choices` *before* type conversion happens seems more sensible to me and would fix this, as well  fullfil Victor's original usecase (though not with the syntax he suggests).
msg323681 - (view) Author: Matthijs Kooijman (Matthijs Kooijman) Date: 2018-08-17 22:13
One way to make the original example from Victor work, is to use the following action class:

  class ConvertChoices(argparse.Action):                                                                                                                     
      Argparse action that interprets the `choices` argument as a dict                                                    
      mapping the user-specified choices values to the resulting option                                                   
      def __init__(self, *args, choices, **kwargs):                                                                       
          super().__init__(*args, choices=choices.keys(), **kwargs)                                                       
          self.mapping = choices                                                                                                                             
      def __call__(self, parser, namespace, value, option_string=None):                                                   
          setattr(namespace, self.dest, self.mapping[value])
msg323732 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2018-08-18 20:55
The 'choices' mechanism is quite simple.  As noted for testing it simply uses an 'in' test.  

For help formating it uses

     choice_strs = [str(choice) for choice in action.choices]
     result = '{%s}' % ','.join(choice_strs)

In other words, it's treated as a iterable. 

But it is easy to produce unmanageable displays, such as with 'range(100)'.  This has been raised in other bug/issues.  The best way around that is with the 'metavar', which lets you customize the 'usage' and 'help'.  One thing that 'metavar' does not help with is the error display.

I'm not privy to the original author's thinking, but I don't think 'choices' was ever meant to be a high power tool.  

With a custom Action you can do almost anything that you could do after parsing.
Date User Action Args
2018-09-23 01:24:49paul.j3setstatus: open -> closed
resolution: rejected
stage: resolved
2018-08-18 20:55:45paul.j3setmessages: + msg323732
2018-08-17 22:13:52Matthijs Kooijmansetmessages: + msg323681
2018-08-17 21:42:19Matthijs Kooijmansetnosy: + Matthijs Kooijman
messages: + msg323673
2018-07-23 16:46:22paul.j3setnosy: + paul.j3
messages: + msg322233
2018-07-22 22:18:49rhettingersetmessages: + msg322170
2018-07-22 21:33:33portonsetmessages: + msg322168
2018-07-22 20:18:50rhettingersetversions: + Python 3.8, - Python 3.7
nosy: + rhettinger, bethard

messages: + msg322167

assignee: bethard
type: behavior -> enhancement
2018-07-22 16:57:06ppperrysettitle: Use dicts to "transform" argparse arguments to values -> Allow dict choices to "transform" values in argpagse
2018-07-22 13:31:54zach.waresetnosy: + zach.ware
messages: + msg322144
2018-07-22 12:53:56portonsettype: behavior
2018-07-22 12:53:50portoncreate