classification
Title: In argparse action append_const doesn't work for positional arguments
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.6, Python 3.3
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: paul.j3, py.user
Priority: normal Keywords:

Created on 2015-06-09 23:05 by py.user, last changed 2016-06-21 02:55 by paul.j3. This issue is now closed.

Messages (10)
msg245099 - (view) Author: py.user (py.user) * Date: 2015-06-09 23:05
Action append_const works for options:

>>> import argparse
>>> 
>>> parser = argparse.ArgumentParser()
>>> _ = parser.add_argument('--foo', dest='x', action='append_const', const=42)
>>> _ = parser.add_argument('--bar', dest='x', action='append_const', const=43)
>>> parser.parse_args('--foo --bar'.split())
Namespace(x=[42, 43])
>>>

Action append_const works for single positionals:

>>> import argparse
>>> 
>>> parser = argparse.ArgumentParser()
>>> _ = parser.add_argument('foo', action='append_const', const=42)
>>> _ = parser.add_argument('bar', action='append_const', const=43)
>>> parser.parse_args([])
Namespace(bar=[43], foo=[42])
>>>

Action append_const doesn't work for positionals in one list:

>>> import argparse
>>> 
>>> parser = argparse.ArgumentParser()
>>> _ = parser.add_argument('foo', dest='x', action='append_const', const=42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.3/site-packages/argparse-1.1-py3.3.egg/argparse.py", line 1282, in add_argument
    """
ValueError: dest supplied twice for positional argument
>>> _ = parser.add_argument('bar', dest='x', action='append_const', const=43)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.3/site-packages/argparse-1.1-py3.3.egg/argparse.py", line 1282, in add_argument
    """
ValueError: dest supplied twice for positional argument
>>> parser.parse_args([])
Namespace()
>>>


The reason is that a positional argument can't accept dest:

>>> import argparse
>>> 
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('foo', dest='x')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.3/site-packages/argparse-1.1-py3.3.egg/argparse.py", line 1282, in add_argument
    """
ValueError: dest supplied twice for positional argument
>>>
msg245449 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2015-06-18 01:17
What are you trying to accomplish in the examples with a 'dest'?  For a positional, 'dest' is derived from the 'foo' name.  There is no need to supply 'dest', in fact produces the error you get.  It has nothing to with this action type (as your last example demonstrates).

Your case with 'foo' and 'bar' shows that 'append_const' works fine.  

But be ware that such an action does not make much sense for a positional.  Such an action takes no arguments, i.e. 'nargs=0'.  So such a positional is always present, since it does not consume any `argv` strings.  You might even get an error if you supply a string.  (same would be true of 'store_true' and related actions).
msg245450 - (view) Author: py.user (py.user) * Date: 2015-06-18 02:16
paul j3 wrote:
> What are you trying to accomplish in the examples with a 'dest'?

To append all that constants to one list.

From this:
Namespace(bar=[43], foo=[42])

To this:
Namespace(x=[43, 42])
msg245451 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2015-06-18 02:35
None of the `append` actions makes sense with positionals.  The name (and hence the 'dest') must be unique.  And positionals can't be repeated.

There are other ways to put a pair of values in the Namespace.  For example, after parsing

    args.x = [42, 43]

or before

    ns = argparse.Namespace(x=[42,43])
    parser.parse_args(namespace=ns)

or the `const` (or better the default) could be `[42, 43]`.

Plain `append` might let you put `[42,43]` in the dest via the default, and then append further values to that list (from the user).  I'd have to test that.

It might be instructive to look at the `test_argparse.py` file, and search for test cases that use `append` or `const`.
msg245452 - (view) Author: py.user (py.user) * Date: 2015-06-18 03:12
paul j3 wrote:
> The name (and hence the 'dest') must be unique.

The problem is in the dest argument of add_argument(). Why user can't set a custom name for a positional?

We can use this list not only for positionals but for optionals too.
msg245473 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2015-06-18 17:01
You can give the positional any custom name (the first parameter).  You just can't reuse it (by giving 2 positionals the same name).  And if you don't like what the 'help' shows, you can set the 'metavar'.  That way only you see the positional's name.

The name of a positional can be the 'dest' of an optional.  But wouldn't that be confusing?  Setting the same attribute with a required postional and one or more optional optionals?

'nargs' is another way of assigning more than one value to a Namespace attribute.  You just can't put an optional between two such values.

`argparse` is a parser, a way of identifying what the user gives you.  It is better to err on the side of preserving information.  Different argument dests does that.   You can always combine values after parsing.

    args.foo.append(args.bar)   # or .extend()
    args.x = [args.foo, args.bar]

Don't try to force argparse to do something special when you can just as easily do that later with normal Python expressions.
msg245477 - (view) Author: py.user (py.user) * Date: 2015-06-18 19:29
paul j3 wrote:
> You can give the positional any custom name (the first parameter).

The dest argument is not required for giving name for an optional.
You can either make it automatically or set by dest, it's handy and clear.

>>> import argparse
>>> 
>>> parser = argparse.ArgumentParser()
>>> _ = parser.add_argument('-a', '--aa')
>>> _ = parser.add_argument('-b', '--bb', dest='x')
>>> args = parser.parse_args([])
>>> print(args)
Namespace(aa=None, x=None)
>>>

But if you do the same thing with a positional, it throws an exception. Why?
(I'm a UNIX user and waiting predictable behaviour.)

And the situation with another action (not only append_const, but future extensions) shows that dest may be required.
msg245482 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2015-06-18 20:49
(Important correction at the end of this post)

The test that you are complaining about occurs at the start of the 'add_argument' method:

    def add_argument(self, *args, **kwargs):
        """
        add_argument(dest, ..., name=value, ...)
        add_argument(option_string, option_string, ..., name=value, ...)
        """

        # if no positional args are supplied or only one is supplied and
        # it doesn't look like an option string, parse a positional
        # argument
        chars = self.prefix_chars
        if not args or len(args) == 1 and args[0][0] not in chars:
            if args and 'dest' in kwargs:
                raise ValueError('dest supplied twice for positional argument')
            kwargs = self._get_positional_kwargs(*args, **kwargs)

        # otherwise, we're adding an optional argument
        else:
            kwargs = self._get_optional_kwargs(*args, **kwargs)
        ...

and the 2 methods it calls:

    def _get_positional_kwargs(self, dest, **kwargs):
        # code to deduce the 'required' parameter ...
        # return the keyword arguments with no option strings
        return dict(kwargs, dest=dest, option_strings=[])

    def _get_optional_kwargs(self, *args, **kwargs):
        # determine short and long option strings
        ....
        # infer destination, '--foo-bar' -> 'foo_bar' and '-x' -> 'x'
        dest = kwargs.pop('dest', None)
        if dest is None:
            if long_option_strings:
                dest_option_string = long_option_strings[0]
            else:
                dest_option_string = option_strings[0]
            dest = dest_option_string.lstrip(self.prefix_chars)
            if not dest:
                msg = _('dest= is required for options like %r')
                raise ValueError(msg % option_string)
            dest = dest.replace('-', '_')

        # return the updated keyword arguments
        return dict(kwargs, dest=dest, option_strings=option_strings)

At the 'add_argument' stage, a big difference between positionals and optionals is in how 'dest' is deduced.  Note the doc string.

During parsing, positionals are distinguished from optionals by the 'option_strings' attribute (empty or not).  'dest' is not used during parsing, except by the Action '__call__'.

-------------------------

I just thought of another way around this constraint - set 'dest' after the action is created:

    p=argparse.ArgumentParser()
    a1=p.add_argument('foo',action='append')
    a2=p.add_argument('bar',action='append')
    a1.dest='x'
    a2.dest='x'
    args=p.parse_args(['one','two'])

produces

    Namespace(x=['one', 'two'])

This works because after the action has been created, no one checks whether the 'dest' value is duplicated or even looks pretty (except when trying to format it for the help.

You could also write a custom Action class, one that mangles the 'dest' to your heart's delight.  The primary use of 'self.dest' is in the expression:

    setattr(namespace, self.dest, items)

you could replace this line in the Action '__call__' with

    setattr(namespace, 'secret#dest', items)

-----------------

I was mistaken on one thing - you can reuse positional 'names':

     a1=p.add_argument('foo',action='append')
     a2=p.add_argument('foo',action='append',type=int)
     p.parse_args(['a','3'])

produces:

     Namespace(foo=['a', 3])

There is a 'name' conflict handler, but it only pays attention to the option strings (flags for optionals).  You can't have two arguments using '-f' or '--foo'.  But you can have 2 or more positionals with the same 'dest'.  You just have to set the dest the right way.

This last point renders the whole issue moot.  But I'll leave it at the end to reflect my train of thought.
msg245641 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2015-06-22 17:36
To wrap this up, the correct way to specify that 2 or more positionals share a 'dest' is to supply that dest as the first parameter.  If the help should have something else, use the `metavar`.

    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument('x', action='append_const', const=42, metavar='foo')
    parser.add_argument('x', action='append_const', const=43, metavar='bar')
    parser.print_help()
    args=parser.parse_args([])
    print(args)

produces

    usage: issue24419.py [-h]

    positional arguments:
      foo
      bar

    optional arguments:
      -h, --help  show this help message and exit
    Namespace(x=[42, 43])

(I think this issue can be closed).
msg245659 - (view) Author: py.user (py.user) * Date: 2015-06-22 23:14
Tested on argdest.py:

#!/usr/bin/env python3

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('x', action='append')
parser.add_argument('x', action='append_const', const=42, metavar='foo')
parser.add_argument('x', action='append_const', const=43, metavar='bar')
parser.add_argument('-x', action='append_const', const=44)

args = parser.parse_args()
print(args)


Run:

[guest@localhost debug]$ ./argdest.py -h
usage: argdest.py [-h] [-x] x

positional arguments:
  x
  foo
  bar

optional arguments:
  -h, --help  show this help message and exit
  -x
[guest@localhost debug]$ ./argdest.py -x 1 -x
Namespace(x=[44, '1', 42, 43, 44])
[guest@localhost debug]$


LGTM.
History
Date User Action Args
2016-06-21 02:55:14paul.j3setstatus: open -> closed
2015-06-22 23:14:56py.usersetmessages: + msg245659
2015-06-22 17:36:05paul.j3setmessages: + msg245641
2015-06-18 20:49:42paul.j3setmessages: + msg245482
2015-06-18 19:29:00py.usersetmessages: + msg245477
2015-06-18 17:01:51paul.j3setmessages: + msg245473
2015-06-18 03:12:30py.usersetmessages: + msg245452
2015-06-18 02:35:43paul.j3setmessages: + msg245451
2015-06-18 02:16:56py.usersetmessages: + msg245450
2015-06-18 01:17:23paul.j3setnosy: + paul.j3
messages: + msg245449
2015-06-09 23:05:09py.usercreate