Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zipfile.Path.iterdir() outputs sub directories many times or not at all #81953

Closed
joernheissler mannequin opened this issue Aug 6, 2019 · 7 comments
Closed

zipfile.Path.iterdir() outputs sub directories many times or not at all #81953

joernheissler mannequin opened this issue Aug 6, 2019 · 7 comments
Assignees
Labels
3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@joernheissler
Copy link
Mannequin

joernheissler mannequin commented Aug 6, 2019

BPO 37772
Nosy @jaraco, @joernheissler, @shireenrao
PRs
  • bpo-37772: fix zipfile.Path.iterdir() outputs #15170
  • [3.8] bpo-37772: fix zipfile.Path.iterdir() outputs (GH-15170) #15461
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/jaraco'
    closed_at = <Date 2019-08-24.16:28:26.115>
    created_at = <Date 2019-08-06.06:57:30.618>
    labels = ['3.8', 'type-bug', 'library', '3.9']
    title = 'zipfile.Path.iterdir() outputs sub directories many times or not at all'
    updated_at = <Date 2019-08-24.16:28:26.102>
    user = 'https://github.com/joernheissler'

    bugs.python.org fields:

    activity = <Date 2019-08-24.16:28:26.102>
    actor = 'jaraco'
    assignee = 'jaraco'
    closed = True
    closed_date = <Date 2019-08-24.16:28:26.115>
    closer = 'jaraco'
    components = ['Library (Lib)']
    creation = <Date 2019-08-06.06:57:30.618>
    creator = 'joernheissler'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 37772
    keywords = ['patch']
    message_count = 7.0
    messages = ['349101', '349150', '349201', '349205', '350374', '350375', '350376']
    nosy_count = 3.0
    nosy_names = ['jaraco', 'joernheissler', 'shireenrao']
    pr_nums = ['15170', '15461']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue37772'
    versions = ['Python 3.8', 'Python 3.9']

    @joernheissler
    Copy link
    Mannequin Author

    joernheissler mannequin commented Aug 6, 2019

    Hello,

    #!/usr/bin/python3.8

    from zipfile import ZipFile, Path
    import io
    
    def recurse_print(parent):
        for child in parent.iterdir():
            if child.is_file():
                print(child, child.read_text())
            if child.is_dir():
                recurse_print(child)
    
    data = io.BytesIO()
    zf = ZipFile(data, "w")
    zf.writestr("a.txt", "content of a")
    zf.writestr("b/c.txt", "content of c")
    zf.writestr("b/d/e.txt", "content of e")
    zf.writestr("b/f.txt", "content of f")
    zf.filename = "abcde.zip"
    root = Path(zf)
    recurse_print(root)

    Expected result:

    abcde.zip/a.txt content of a
    abcde.zip/b/c.txt content of c
    abcde.zip/b/f.txt content of f
    abcde.zip/b/d/e.txt content of e

    Actual result:

    abcde.zip/a.txt content of a
    abcde.zip/b/c.txt content of c
    abcde.zip/b/f.txt content of f
    abcde.zip/b/d/e.txt content of e
    abcde.zip/b/c.txt content of c
    abcde.zip/b/f.txt content of f
    abcde.zip/b/d/e.txt content of e

    Path._add_implied_dirs adds the sub directory "b/" twice: once for each direct child (i.e. "c.txt" and "f.txt")

    And similarly:

    data = io.BytesIO()
    zf = ZipFile(data, "w")
    zf.writestr("a.txt", "content of a")
    zf.writestr("b/d/e.txt", "content of e")
    zf.filename = "abcde.zip"
    root = Path(zf)
    recurse_print(root)

    Expected result:

    abcde.zip/a.txt content of a
    abcde.zip/b/d/e.txt content of e

    Actual result:

    abcde.zip/a.txt content of a

    Here, Path._add_implied_dirs doesn't add "b/" at all, because there are no direct childs of "b".

    @joernheissler joernheissler mannequin added 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Aug 6, 2019
    @shireenrao
    Copy link
    Mannequin

    shireenrao mannequin commented Aug 7, 2019

    I have attempted a fix to Path._add_implied_dirs and have the changes in my github branch https://github.com/shireenrao/cpython/tree/zipfile. Is it ok if I submitted a PR on this? I have run test_zipfile.py and the test case provided by Jörn Heissler successfully.

    @jaraco
    Copy link
    Member

    jaraco commented Aug 7, 2019

    Please do.

    @jaraco jaraco self-assigned this Aug 7, 2019
    @shireenrao
    Copy link
    Mannequin

    shireenrao mannequin commented Aug 8, 2019

    I just submitted my PR.

    @jaraco
    Copy link
    Member

    jaraco commented Aug 24, 2019

    New changeset a4e2991 by Jason R. Coombs (shireenrao) in branch 'master':
    bpo-37772: fix zipfile.Path.iterdir() outputs (GH-15170)
    a4e2991

    @jaraco
    Copy link
    Member

    jaraco commented Aug 24, 2019

    New changeset c410f38 by Jason R. Coombs (Miss Islington (bot)) in branch '3.8':
    bpo-37772: fix zipfile.Path.iterdir() outputs (GH-15170) (bpo-15461)
    c410f38

    @jaraco
    Copy link
    Member

    jaraco commented Aug 24, 2019

    Thanks very much shireenrao for the PR, which is now merged with Python 3.8+. I've additionally backported the fix to zipp in 0.6.0.

    @jaraco jaraco added the 3.8 only security fixes label Aug 24, 2019
    @jaraco jaraco closed this as completed Aug 24, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant