This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: argparse: action "extend" with 1 parameter splits strings into characters
Type: behavior Stage:
Components: Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Anthony Sottile, BTaskaya, Jonathan Haigh, berker.peksag, paul.j3, rhettinger, strjan
Priority: normal Keywords:

Created on 2020-04-22 17:42 by strjan, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
argparse_bug.py strjan, 2020-04-22 17:42 Minimal example
Messages (4)
msg367033 - (view) Author: (strjan) Date: 2020-04-22 17:42
When positional argument with action='extend' is given to parse only one string argument, the string is splitted into individual characters.

The problem occures, when no "nargs" is given; it makes sense that nargs is requiered, though at least notion in documentation would be nice, as the issue does not occure with action='append'

Minimal example included.
msg367045 - (view) Author: paul j3 (paul.j3) * (Python triager) Date: 2020-04-22 19:26
This is a consequence of Python's own definition of append vs extend

In [730]: alist = []                                                                                   
In [731]: alist.append('astring')                                                                      
In [732]: alist                                                                                        
Out[732]: ['astring']
In [733]: alist.extend('astring')                                                                      
In [734]: alist                                                                                        
Out[734]: ['astring', 'a', 's', 't', 'r', 'i', 'n', 'g']
In [735]: alist.extend(['astring'])                                                                    
In [736]: alist                                                                                        
Out[736]: ['astring', 'a', 's', 't', 'r', 'i', 'n', 'g', 'astring']

Normally 'add_argument' doesn't check for valid parameters, but some Action subclasses do their own checking, 

'extend' inherits the nargs==0 test from 'append'. (extend just modifies the __call__, not the __init__ method.

But I wonder, was this situation discussed in the original bug/issue?
msg370898 - (view) Author: Jonathan Haigh (Jonathan Haigh) * Date: 2020-06-07 13:36
The situation for type=int and unspecified nargs or nargs="?" is also surprising: 

Python 3.8.3 (default, May 21 2020, 12:19:36) 
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import argparse
>>> p = argparse.ArgumentParser()
>>> p.add_argument("--c", action="extend", type=int)
_ExtendAction(option_strings=['--c'], dest='c', nargs=None, const=None, default=None, type=<class 'int'>, choices=None, help=None, metavar=None)
>>> p.parse_args("--c 1".split())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1768, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1800, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 2006, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1946, in consume_optional
    take_action(action, args, option_string)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1874, in take_action
    action(self, namespace, argument_values, option_string)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1171, in __call__
    items.extend(values)
TypeError: 'int' object is not iterable
>>> p = argparse.ArgumentParser()
>>> p.add_argument("--c", action="extend", type=int, nargs="?")
_ExtendAction(option_strings=['--c'], dest='c', nargs='?', const=None, default=None, type=<class 'int'>, choices=None, help=None, metavar=None)
>>> p.parse_args("--c 1".split())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1768, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1800, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 2006, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1946, in consume_optional
    take_action(action, args, option_string)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1874, in take_action
    action(self, namespace, argument_values, option_string)
  File "/home/jonathan/.pyenv/versions/3.8.3/lib/python3.8/argparse.py", line 1171, in __call__
    items.extend(values)
TypeError: 'int' object is not iterable
>>>

I suggest that the default nargs for extend should be "*" or "+" and an exception should be raised if nargs is given as "?". I don't see the current behaviour with unspecified nargs or nargs="?" being useful (and it certainly is surprising). In both cases, I think the least surprising behaviour would be for extend to act the same as append (or for an exception to be raised).

> But I wonder, was this situation discussed in the original bug/issue?
Doesn't look like it:
https://bugs.python.org/issue23378
https://github.com/python/cpython/commit/aa32a7e1116f7aaaef9fec453db910e90ab7b101
msg371103 - (view) Author: Jonathan Haigh (Jonathan Haigh) * Date: 2020-06-09 13:20
>> But I wonder, was this situation discussed in the original bug/issue?
>Doesn't look like it:

I was looking at the wrong PR link. This has more discussion: https://github.com/python/cpython/pull/13305.

nargs is discussed but I'm not sure it was realized that the nargs=None and nargs="?" cases would act in the way seen here rather than acting like append.

Having a default nargs of "+" was suggested but that suggestion was not addressed.

> I suggest that the default nargs for extend should be "*" or "+" and an exception should be raised if nargs is given as "?".

I'm not convinced about that any more. Using append's behaviour is probably more reasonable for nargs=None and nargs="?".
History
Date User Action Args
2022-04-11 14:59:29adminsetgithub: 84545
2020-06-09 13:20:45Jonathan Haighsetnosy: + berker.peksag, Anthony Sottile, BTaskaya
messages: + msg371103
2020-06-07 13:36:51Jonathan Haighsetnosy: + Jonathan Haigh
messages: + msg370898
2020-04-22 19:26:48paul.j3setmessages: + msg367045
2020-04-22 18:49:00xtreaksetnosy: + rhettinger, paul.j3
2020-04-22 17:42:36strjancreate