classification
Title: 2to3 does not wrap zip correctly
Type: behavior Stage:
Components: 2to3 (2.x to 3.x conversion tool) Versions: Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, cvk, stuarteberg
Priority: normal Keywords: patch

Created on 2016-11-30 11:04 by cvk, last changed 2017-02-15 18:30 by stuarteberg.

Files
File name Uploaded Description Edit
fix-28837.patch stuarteberg, 2017-01-19 18:02 In lib2to3 fixers for map, zip, and filter, handle extra "trailers", e.g. zip()[0] review
Pull Requests
URL Status Linked Edit
PR 24 stuarteberg, 2017-02-15 18:30
Messages (4)
msg282075 - (view) Author: Carsten (cvk) Date: 2016-11-30 11:04
The following Python 2.7 code is not converted correctly to Python 3 by 2to3:

c = [(1, 10), (2, 20)]

# get a tuple with the first entries of every tuple in c,
# i.e. (1, 2)
x = zip(*c)[0]

print x


The result is


c = [(1, 10), (2, 20)]

# get a tuple with the first entries of every tuple in c,
# i.e. (1, 2)
x = zip(*c)[0]

print(x)



However running it with python3 gives the following exception

Traceback (most recent call last):
  File "result.py", line 7, in <module>
    x = zip(*c)[0]
TypeError: 'zip' object is not subscriptable



Tested with 2to3-2.7 and 2to3-3.4 on debian stable (jessie)
msg285796 - (view) Author: Stuart Berg (stuarteberg) * Date: 2017-01-19 15:08
In addition to zip(), this problem also affects map() and filter().

The problem is that the match patterns in FixZip, FixMap, and FixFilter do not allow for more than one "trailer" node.  (And even if they did, their transform() methods aren't expecting it.)

For example, in the following expression, 'zip' is followed by two 'trailers', which are '(a,b)', and [0]:

zip(a,b)[0]

... but FixZip.PATTERN only expects a single trailer (the argument list), so the presence of a second trailer prevents the match: https://git.io/vMDP9


(Here's the relevant line of the grammar: https://git.io/vMDPJ)

I've written a patch that fixes this problem for zip, map, and filter, with tests.  See attached.

BTW, this problem was previously reported in 21628, but that issue was incorrectly closed as a duplicate, so I'm submitting my patch here.
msg285823 - (view) Author: Stuart Berg (stuarteberg) * Date: 2017-01-19 17:53
Sorry for re-uploading the patch; I made some pep8 fixes.
msg287596 - (view) Author: Stuart Berg (stuarteberg) * Date: 2017-02-11 08:42
Patch submitted as github PR #24:
https://github.com/python/cpython/pull/24
History
Date User Action Args
2017-02-15 18:30:06stuartebergsetpull_requests: + pull_request85
2017-02-11 08:42:30stuartebergsetmessages: + msg287596
2017-01-19 18:02:26stuartebergsetfiles: + fix-28837.patch
2017-01-19 18:01:51stuartebergsetfiles: - fix-28837.patch
2017-01-19 17:53:48stuartebergsetfiles: - fix-28837.patch
2017-01-19 17:53:39stuartebergsetfiles: + fix-28837.patch

messages: + msg285823
2017-01-19 17:39:29stuartebergsetfiles: + fix-28837.patch
2017-01-19 17:37:46stuartebergsetfiles: - fix-28837.patch
2017-01-19 15:08:08stuartebergsetfiles: + fix-28837.patch

nosy: + stuarteberg
messages: + msg285796

keywords: + patch
2016-12-01 03:02:02xiang.zhangsetnosy: + benjamin.peterson

versions: + Python 3.6, Python 3.7, - Python 3.4
2016-11-30 11:04:58cvkcreate