classification
Title: FastPath.zip_children can give duplicate results on Python 3.8
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: jaraco Nosy List: barry, jaraco
Priority: normal Keywords: patch

Created on 2020-09-24 20:30 by jaraco, last changed 2020-10-15 21:05 by jaraco.

Pull Requests
URL Status Linked Edit
PR 22403 closed jaraco, 2020-09-24 20:43
PR 22404 merged jaraco, 2020-09-24 22:17
Messages (3)
msg377461 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2020-09-24 20:30
Scott J. reports in an e-mail:

When upgrading to Python 3.8.2, we noticed two issues in
importlib.metadata which are not present in importlib_metadata.

1.  FastPath.zip_children() is very slow for large zips. Python 3.8.3
    > already has a fix for this
    > ([[bug]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.python.org%2Fissue39667&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462361570&sdata=Msk47lScorCQpx5PHiKrpeYYzBLMR89HNZ7RwY7ch4c%3D&reserved=0),
    > [[changelog]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.python.org%2F3%2Fwhatsnew%2Fchangelog.html%23id45&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462371564&sdata=CSPBVi5rPFSCovRZdd8WNbOQ4mxDycy%2FmjDhbty4OLk%3D&reserved=0)),
    > which includes merging in importlib_metadata 1.5.0.

2.  FastPath.zip_children() can give duplicate results, causing
    > duplicate results in entry_points(). This issue was reported in
    > importlib_metadata in March
    > ([[link]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fpython-devs%2Fimportlib_metadata%2F-%2Fissues%2F117&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462371564&sdata=g5N0ew3WAOUfPJ5eL7oY0j9NIpXkh8HAkV%2FWD4UppX8%3D&reserved=0))
    > and fixed in importlib_metadata 1.5.2
    > ([[changelog]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fimportlib-metadata.readthedocs.io%2Fen%2Flatest%2Fchangelog.html%23v1-5-2&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462381558&sdata=UpnqBoyl%2FcGMkF%2BfFufredA0QGt31o5Qag3joQ%2FJhDs%3D&reserved=0)).

In June, importlib_metadata 1.6.1 (including the fix for \#2) was merged
into Python
([[bug]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.python.org%2Fissue39791%23msg370782&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462381558&sdata=i8Y8o5f3uBXPP8OU9I3PkKSHzS%2B3z4kcd%2FdH1l%2B9YI8%3D&reserved=0),
[[PR]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F20659&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462391552&sdata=Q5P3RuTNpctc4E93l2dqXRc1LYmw9XSnBb%2FaJbKTcMg%3D&reserved=0))
and backported to 3.9
([[PR]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F20661&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462391552&sdata=f%2BghmbplIsbmc9CFgoQX7vxc%2FZPLG2TkEgjMZtez71s%3D&reserved=0)).
However, the backport for Python 3.8 was **not** merged
([[PR]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F20662&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462391552&sdata=JTpQpeuzE%2BGvL%2FNccWcuu0vQWRKQbkBHLmzeNmWZMFc%3D&reserved=0)).
Jason said:

> Of course this can\'t backport to 3.8; importlib.resources.files
> doesn\'t exist there. No problem. Bug fixes will need to be backported
> specially if needed.

As far as I can tell, no version of importlib_metadata newer than 1.5.0
has ever been merged into Python 3.8, and so bug \#2 is still present
(tested in 3.8.6rc1). This seems like it falls under the \"bug fixes if
needed\" category. We just need to fix the duplicate entries, as
reported in importlib_metadata. This should be trivial. The diff in
python 3.9 is
[[here]{.ul}](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F20661%2Ffiles%23diff-499abe3a411df5cf55659b640ac3b2b4L411-R435&data=02%7C01%7Cbwarsaw%40linkedin.com%7C514c7c4d1eea417ed26908d86024cd43%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637365058462401547&sdata=qZXa9e13SVng5tkL3Im2s1Gc2OSLOKlqiiDKBTqkxoI%3D&reserved=0).
msg377462 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2020-09-24 20:31
The relevant commit is already present in importlib_metadata as [079ca1cb701a5f4aab0a9cb00b0782c7ea8fb70b](https://gitlab.com/python-devs/importlib_metadata/-/commit/079ca1cb701a5f4aab0a9cb00b0782c7ea8fb70b).
msg378699 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2020-10-15 21:05
New changeset 967fddae2fe48f297563c358bdbdde1e2cfed4ee by Jason R. Coombs in branch '3.8':
[3.8] bpo-41855: Fix duplicate results in FastPath.zip_children() (#22404)
https://github.com/python/cpython/commit/967fddae2fe48f297563c358bdbdde1e2cfed4ee
History
Date User Action Args
2020-10-15 21:05:20jaracosetmessages: + msg378699
2020-09-24 22:17:54jaracosetpull_requests: + pull_request21444
2020-09-24 20:43:10jaracosetkeywords: + patch
stage: patch review
pull_requests: + pull_request21443
2020-09-24 20:31:51jaracosetmessages: + msg377462
components: + Library (Lib)
2020-09-24 20:30:47jaracocreate