classification
Title: Regression for star argument parameter error messages
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: 27358 Superseder:
Assigned To: serhiy.storchaka Nosy List: Demur Rumed, haypo, kayhayen, larry, ned.deily, python-dev, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-09-23 11:55 by kayhayen, last changed 2017-03-31 16:36 by dstufft. This issue is now closed.

Files
File name Uploaded Description Edit
built_tuple_unpack_with_call.patch serhiy.storchaka, 2016-09-28 08:09 review
build-map-unpack-with-call-error-messages-3.5.patch serhiy.storchaka, 2016-10-02 08:37 review
Pull Requests
URL Status Linked Edit
PR 552 closed dstufft, 2017-03-31 16:36
Messages (11)
msg277270 - (view) Author: Kay Hayen (kayhayen) Date: 2016-09-23 11:55
Hello,

there is a regression in the beta (alpha 4 was ok) for this kind of code:

print("Complex call with both invalid star list and star arguments:")

try:
    a = 1
    b = 2.0

    functionWithDefaults(1,c = 3,*a,**b)
except TypeError as e:
    print(repr(e))

try:
    a = 1
    b = 2.0

    functionWithDefaults(1,*a,**b)
except TypeError as e:
    print(repr(e))

try:
    a = 1
    b = 2.0

    functionWithDefaults(c = 1, *a,**b)
except TypeError as e:
    print(repr(e))

try:
    a = 1
    b = 2.0

    functionWithDefaults(*a,**b)
except TypeError as e:
    print(repr(e))

This prints with beta1 3.6

Complex call with both invalid star list and star arguments:
TypeError("'int' object is not iterable",)
TypeError("'int' object is not iterable",)
TypeError("'float' object is not iterable",)
TypeError('functionWithDefaults() argument after ** must be a mapping, not float',)

The later message is what they all probably should be like. This is 3.5 output:

Complex call with both invalid star list and star arguments:
TypeError('functionWithDefaults() argument after ** must be a mapping, not float',)
TypeError('functionWithDefaults() argument after ** must be a mapping, not float',)
TypeError('functionWithDefaults() argument after ** must be a mapping, not float',)
TypeError('functionWithDefaults() argument after ** must be a mapping, not float',)

The function itself doesn't matter obviously, it's never called. Please restore the old behavior, thanks.

Yours,
Kay
msg277278 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-23 14:06
This is a consequence of issue27213. Actually there are two issues: with var-positional and var-keyword arguments.

But Python 3.5 is not consistent. It raises an exception with less detailed message if there are multiple var-positional or var-keyword arguments.

Var-positional arguments:

>>> f(*0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after * must be an iterable, not int
>>> f(1, *0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after * must be an iterable, not int
>>> f(*[], *0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable

Python 3.6 just raises the latter message in case of positional arguments and single var-positional argument.

>>> f(*0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after * must be an iterable, not int
>>> f(1, *0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
>>> f(*[], *0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable

This issue can't be fixed without adding new bytecode (BUILD_TUPLE_UNPACK_WITH_CALL). If it will be decided to fix it in 3.6, it may be worth to backport this to 3.5.

Var-keyword arguments:

Python 3.5:

>>> f(**[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after ** must be a mapping, not list
>>> f(x=1, **0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after ** must be a mapping, not int
>>> f(x=1, **[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after ** must be a mapping, not list
>>> f(**{}, **0)   
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
>>> f(**{}, **[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'list' object is not a mapping

Python 3.6 raises less detailed error message in case of keyword arguments and single var-keyword argument.

>>> f(**0)                      
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after ** must be a mapping, not int
>>> f(**[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after ** must be a mapping, not list
>>> f(x=1, **0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable
>>> f(x=1, **[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'list' object is not a mapping
>>> f(**{}, **0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not a mapping
>>> f(**{}, **[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'list' object is not a mapping

This issue can be fixed without changing bytecode. The patch faster_build_map_unpack_with_call.patch for issue27358 fixes it.
msg277282 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-23 15:15
Ned and Larry.

Is it allowed to use the patch from issue27358 for fixing a regression in 3.6 and inconsistency in 3.5 for var-keyword arguments? The patch introduces new private function _PyDict_MergeEx() and touches the implementation of PyDict_Merge(). Maybe this can be fixed with smaller patch, but I don't know.

Is it allowed to add new opcode BUILD_TUPLE_UNPACK_WITH_CALL for fixing a regression in 3.6 and inconsistency in 3.5 for var-positional arguments? This is the only way. The bytecode already was changed in a bugfix release of 3.5 for more serious need: fixing potential segfault or security issue (issue27286).
msg277284 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-09-23 15:35
It's too late to change this for 3.5.
msg277289 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-09-23 17:53
That is a fairly big change to go in now but the error message regression is also big.  With a review from a core developer and assuming no other objections, I think this can go into 360b2.
msg277293 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-09-23 19:13
> It's too late to change this for 3.5.

The whole point of having a beta release is to detect issues like this.
msg277294 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-09-23 19:14
Raymond, Larry's comment was about 3.5, not 3.6.  See my comment above.
msg277594 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-28 08:09
Proposed patch fixes error message for var-positional arguments. It adds new opcode BUILD_TUPLE_UNPACK_WITH_CALL.

>>> min(1, *2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: min() argument after * must be an iterable, not int
>>> min(*[1], *2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: min() argument after * must be an iterable, not int
msg277858 - (view) Author: Roundup Robot (python-dev) Date: 2016-10-02 07:38
New changeset 88b319dfc909 by Serhiy Storchaka in branch '3.6':
Issue #28257: Improved error message when pass a non-iterable as
https://hg.python.org/cpython/rev/88b319dfc909

New changeset bde594cd8369 by Serhiy Storchaka in branch 'default':
Issue #28257: Improved error message when pass a non-iterable as
https://hg.python.org/cpython/rev/bde594cd8369

New changeset 40d7ce58ebd0 by Serhiy Storchaka in branch '3.5':
Issue #28257: Backported a test.
https://hg.python.org/cpython/rev/40d7ce58ebd0

New changeset a8168a52a56f by Serhiy Storchaka in branch '2.7':
Issue #28257: Backported a test.
https://hg.python.org/cpython/rev/a8168a52a56f
msg277860 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-02 08:37
Here is a patch with smaller (in comparison with issue27358) change for 3.5 that improves error message when pass a non-mapping as second var-keyword argument.

Unpatched:

>>> f(**{'a': 1}, **[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'list' object is not a mapping
>>> f(**{'a': 1}, **0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not iterable

Patched:

>>> f(**{'a': 1}, **[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after ** must be a mapping, not list
>>> f(**{'a': 1}, **0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() argument after ** must be a mapping, not int
msg278268 - (view) Author: Roundup Robot (python-dev) Date: 2016-10-07 20:35
New changeset 35676cd72352 by Serhiy Storchaka in branch '3.5':
Issue #28257: Improved error message when pass a non-mapping as a var-keyword
https://hg.python.org/cpython/rev/35676cd72352
History
Date User Action Args
2017-03-31 16:36:25dstufftsetpull_requests: + pull_request985
2016-10-07 20:36:17serhiy.storchakasetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.6, Python 3.7
2016-10-07 20:35:24python-devsetmessages: + msg278268
2016-10-02 08:37:57serhiy.storchakasetfiles: + build-map-unpack-with-call-error-messages-3.5.patch
priority: deferred blocker -> normal
versions: + Python 3.5, - Python 3.6, Python 3.7
messages: + msg277860

assignee: serhiy.storchaka
2016-10-02 07:38:16python-devsetnosy: + python-dev
messages: + msg277858
2016-09-28 08:09:30serhiy.storchakasetfiles: + built_tuple_unpack_with_call.patch
keywords: + patch
messages: + msg277594

stage: patch review
2016-09-23 19:14:09ned.deilysetmessages: + msg277294
2016-09-23 19:13:00rhettingersetnosy: + rhettinger
messages: + msg277293
2016-09-23 17:53:33ned.deilysetmessages: + msg277289
2016-09-23 15:35:59larrysetnosy: haypo, larry, ned.deily, kayhayen, serhiy.storchaka, Demur Rumed
messages: + msg277284
versions: - Python 3.5
2016-09-23 15:15:30serhiy.storchakasetpriority: normal -> deferred blocker

messages: + msg277282
2016-09-23 14:06:55serhiy.storchakasetversions: + Python 3.5, Python 3.7
nosy: + larry, Demur Rumed, serhiy.storchaka, haypo, ned.deily

messages: + msg277278

dependencies: + BUILD_MAP_UNPACK_WITH_CALL is slow
components: + Interpreter Core
2016-09-23 11:55:46kayhayencreate