Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zlib.error with tarfile.open #83220

Closed
jvoisin mannequin opened this issue Dec 13, 2019 · 9 comments
Closed

zlib.error with tarfile.open #83220

jvoisin mannequin opened this issue Dec 13, 2019 · 9 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@jvoisin
Copy link
Mannequin

jvoisin mannequin commented Dec 13, 2019

BPO 39039
Nosy @tiran, @ethanfurman, @ambv, @jdevries3133
PRs
  • bpo-39039: tarfile raises descriptive exception from zlib.error #27766
  • [3.10] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) #28613
  • [3.9] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) #28614
  • Files
  • crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-09-29.10:58:55.030>
    created_at = <Date 2019-12-13.16:00:48.662>
    labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
    title = 'zlib.error with tarfile.open'
    updated_at = <Date 2021-09-29.10:58:55.028>
    user = 'https://bugs.python.org/jvoisin'

    bugs.python.org fields:

    activity = <Date 2021-09-29.10:58:55.028>
    actor = 'lukasz.langa'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-09-29.10:58:55.030>
    closer = 'lukasz.langa'
    components = ['Library (Lib)']
    creation = <Date 2019-12-13.16:00:48.662>
    creator = 'jvoisin'
    dependencies = []
    files = ['48774']
    hgrepos = []
    issue_num = 39039
    keywords = ['patch']
    message_count = 9.0
    messages = ['358337', '358340', '358341', '399811', '399989', '402834', '402845', '402847', '402848']
    nosy_count = 5.0
    nosy_names = ['christian.heimes', 'ethan.furman', 'lukasz.langa', 'jvoisin', 'jack__d']
    pr_nums = ['27766', '28613', '28614']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue39039'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    @jvoisin
    Copy link
    Mannequin Author

    jvoisin mannequin commented Dec 13, 2019

    The attached file produces the following stacktrace when opened via tarfile.open, on Python 3.7.5rc1:

    $ cat test.py 
    import sys
    import tarfile
    
    tarfile.open(sys.argv[1])
    $ python3 test.py ./crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82
    Traceback (most recent call last):
      File "test.py", line 4, in <module>
        tarfile.open(sys.argv[1])
      File "/usr/lib/python3.7/tarfile.py", line 1573, in open
        return func(name, "r", fileobj, **kwargs)
      File "/usr/lib/python3.7/tarfile.py", line 1645, in gzopen
        t = cls.taropen(name, mode, fileobj, **kwargs)
      File "/usr/lib/python3.7/tarfile.py", line 1621, in taropen
        return cls(name, mode, fileobj, **kwargs)
      File "/usr/lib/python3.7/tarfile.py", line 1484, in __init__
        self.firstmember = self.next()
      File "/usr/lib/python3.7/tarfile.py", line 2289, in next
        tarinfo = self.tarinfo.fromtarfile(self)
      File "/usr/lib/python3.7/tarfile.py", line 1094, in fromtarfile
        buf = tarfile.fileobj.read(BLOCKSIZE)
      File "/usr/lib/python3.7/gzip.py", line 276, in read
        return self._buffer.read(size)
      File "/usr/lib/python3.7/_compression.py", line 68, in readinto
        data = self.read(len(byte_view))
      File "/usr/lib/python3.7/gzip.py", line 471, in read
        uncompress = self._decompressor.decompress(buf, size)
    zlib.error: Error -3 while decompressing data: invalid distances se
    

    @jvoisin jvoisin mannequin added 3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Dec 13, 2019
    @tiran
    Copy link
    Member

    tiran commented Dec 13, 2019

    This file is also an invalid tar file:

    $ tar xf crash-c10c9839d987fa0df6912cb4084f43f3ce08ca82 

    gzip: stdin: invalid compressed data--format violated
    tar: Child returned status 1
    tar: Error is not recoverable: exiting now

    @jvoisin
    Copy link
    Mannequin Author

    jvoisin mannequin commented Dec 13, 2019

    Sure, but as a user, I would expect a better exception, like ValueError or ReadError, along with an error message, instead of an unexpected zlib exception.

    @jdevries3133
    Copy link
    Mannequin

    jdevries3133 mannequin commented Aug 18, 2021

    @jvoisin I am able to reproduce the problem when I download your script, but I am having a hard time reproducing it by passing corrupt archives to tarfile.open. How exactly was this file corrupted? I am trying to figure out if there are any similar implementation leaks / poor error messages in similar scenarios so I can do my best to patch them all.

    You can see the reproduction scripts I am using here to get a better idea of what I have been trying. Be forewarned, they are pretty gnarly!

    https://gist.github.com/jdevries3133/acbb5ba2a19093d3bcc214733ef85e5a

    @jvoisin
    Copy link
    Mannequin Author

    jvoisin mannequin commented Aug 20, 2021

    The file was created with a fuzzer, like the one described in https://dustri.org/b/fuzzing-python-in-python-and-doing-it-fast.html

    @ambv
    Copy link
    Contributor

    ambv commented Sep 29, 2021

    New changeset b6fe857 by Jack DeVries in branch 'main':
    bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766)
    b6fe857

    @ambv
    Copy link
    Contributor

    ambv commented Sep 29, 2021

    New changeset d6b69f2 by Łukasz Langa in branch '3.10':
    [3.10] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) (GH-28613)
    d6b69f2

    @ambv
    Copy link
    Contributor

    ambv commented Sep 29, 2021

    New changeset 7bff4d3 by Łukasz Langa in branch '3.9':
    [3.9] bpo-39039: tarfile raises descriptive exception from zlib.error (GH-27766) (GH-28614)
    7bff4d3

    @ambv
    Copy link
    Contributor

    ambv commented Sep 29, 2021

    Thanks for the fix, Jack! ✨ 🍰 ✨

    Since the change translated zlib.error to tarfile.ReadError which already has to be handled by user code, it's strictly decreasing the surface of necessary exception handling. So, treating this as a bug fix, I backported this to 3.9 and 3.10 as well.

    @ambv ambv added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes and removed 3.7 (EOL) end of life labels Sep 29, 2021
    @ambv ambv closed this as completed Sep 29, 2021
    @ambv ambv added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes and removed 3.7 (EOL) end of life labels Sep 29, 2021
    @ambv ambv closed this as completed Sep 29, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants