classification
Title: zipfile.Path.iterdir() outputs sub directories many times or not at all
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jaraco Nosy List: jaraco, joernheissler, shireenrao
Priority: normal Keywords: patch

Created on 2019-08-06 06:57 by joernheissler, last changed 2019-08-24 16:28 by jaraco. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 15170 merged shireenrao, 2019-08-07 22:49
PR 15461 merged miss-islington, 2019-08-24 15:30
Messages (7)
msg349101 - (view) Author: Jörn Heissler (joernheissler) * Date: 2019-08-06 06:57
Hello,


#!/usr/bin/python3.8

from zipfile import ZipFile, Path
import io

def recurse_print(parent):
    for child in parent.iterdir():
        if child.is_file():
            print(child, child.read_text())
        if child.is_dir():
            recurse_print(child)

data = io.BytesIO()
zf = ZipFile(data, "w")
zf.writestr("a.txt", "content of a")
zf.writestr("b/c.txt", "content of c")
zf.writestr("b/d/e.txt", "content of e")
zf.writestr("b/f.txt", "content of f")
zf.filename = "abcde.zip"
root = Path(zf)
recurse_print(root)


Expected result:

abcde.zip/a.txt content of a
abcde.zip/b/c.txt content of c
abcde.zip/b/f.txt content of f
abcde.zip/b/d/e.txt content of e

Actual result:

abcde.zip/a.txt content of a
abcde.zip/b/c.txt content of c
abcde.zip/b/f.txt content of f
abcde.zip/b/d/e.txt content of e
abcde.zip/b/c.txt content of c
abcde.zip/b/f.txt content of f
abcde.zip/b/d/e.txt content of e


Path._add_implied_dirs adds the sub directory "b/" twice: once for each direct child (i.e. "c.txt" and "f.txt")


And similarly:

data = io.BytesIO()
zf = ZipFile(data, "w")
zf.writestr("a.txt", "content of a")
zf.writestr("b/d/e.txt", "content of e")
zf.filename = "abcde.zip"
root = Path(zf)
recurse_print(root)

Expected result:

abcde.zip/a.txt content of a
abcde.zip/b/d/e.txt content of e

Actual result:

abcde.zip/a.txt content of a

Here, Path._add_implied_dirs doesn't add "b/" at all, because there are no direct childs of "b".
msg349150 - (view) Author: Srinivas Nyayapati (shireenrao) * Date: 2019-08-07 04:25
I have attempted a fix to Path._add_implied_dirs and have the changes in my github branch https://github.com/shireenrao/cpython/tree/zipfile. Is it ok if I submitted a PR on this? I have run test_zipfile.py and the test case provided by Jörn Heissler successfully.
msg349201 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2019-08-07 21:12
Please do.
msg349205 - (view) Author: Srinivas Nyayapati (shireenrao) * Date: 2019-08-08 00:08
I just submitted my PR.
msg350374 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2019-08-24 15:26
New changeset a4e2991bdc993b60b6457c8a38d6e4a1fc845781 by Jason R. Coombs (shireenrao) in branch 'master':
bpo-37772: fix zipfile.Path.iterdir() outputs (GH-15170)
https://github.com/python/cpython/commit/a4e2991bdc993b60b6457c8a38d6e4a1fc845781
msg350375 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2019-08-24 16:03
New changeset c410f381bf66c48d84812e19e3ba7c2878511a3e by Jason R. Coombs (Miss Islington (bot)) in branch '3.8':
bpo-37772: fix zipfile.Path.iterdir() outputs (GH-15170) (#15461)
https://github.com/python/cpython/commit/c410f381bf66c48d84812e19e3ba7c2878511a3e
msg350376 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2019-08-24 16:28
Thanks very much shireenrao for the PR, which is now merged with Python 3.8+. I've additionally backported the fix to zipp in 0.6.0.
History
Date User Action Args
2019-08-24 16:28:26jaracosetstatus: open -> closed
versions: + Python 3.8
messages: + msg350376

resolution: fixed
stage: patch review -> resolved
2019-08-24 16:03:55jaracosetmessages: + msg350375
2019-08-24 15:30:25miss-islingtonsetpull_requests: + pull_request15155
2019-08-24 15:26:44jaracosetmessages: + msg350374
2019-08-08 00:08:16shireenraosetmessages: + msg349205
2019-08-07 22:49:26shireenraosetkeywords: + patch
stage: patch review
pull_requests: + pull_request14902
2019-08-07 21:13:53jaracosetassignee: jaraco
2019-08-07 21:12:45jaracosetmessages: + msg349201
2019-08-07 04:25:14shireenraosetnosy: + shireenrao
messages: + msg349150
2019-08-06 21:03:21brett.cannonsetnosy: + jaraco
2019-08-06 06:57:30joernheisslercreate