Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LZMACompressor and LZMADecompressor raise exceptions if given empty strings twice #71704

Closed
benfogle mannequin opened this issue Jul 15, 2016 · 7 comments
Closed

LZMACompressor and LZMADecompressor raise exceptions if given empty strings twice #71704

benfogle mannequin opened this issue Jul 15, 2016 · 7 comments
Assignees
Labels
3.7 (EOL) end of life extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error

Comments

@benfogle
Copy link
Mannequin

benfogle mannequin commented Jul 15, 2016

BPO 27517
Nosy @serhiy-storchaka, @AraHaan, @benfogle
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • lzma.patch
  • lzma_2.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2016-10-31.06:36:55.434>
    created_at = <Date 2016-07-15.01:03:30.680>
    labels = ['extension-modules', 'type-bug', '3.7']
    title = 'LZMACompressor and LZMADecompressor raise exceptions if given empty strings twice'
    updated_at = <Date 2017-03-31.16:36:25.500>
    user = 'https://github.com/benfogle'

    bugs.python.org fields:

    activity = <Date 2017-03-31.16:36:25.500>
    actor = 'dstufft'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2016-10-31.06:36:55.434>
    closer = 'serhiy.storchaka'
    components = ['Extension Modules']
    creation = <Date 2016-07-15.01:03:30.680>
    creator = 'benfogle'
    dependencies = []
    files = ['43727', '45286']
    hgrepos = []
    issue_num = 27517
    keywords = ['patch']
    message_count = 7.0
    messages = ['270451', '270452', '270455', '279737', '279758', '279764', '279765']
    nosy_count = 5.0
    nosy_names = ['nadeem.vawda', 'python-dev', 'serhiy.storchaka', 'Decorater', 'benfogle']
    pr_nums = ['552']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue27517'
    versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

    @benfogle
    Copy link
    Mannequin Author

    benfogle mannequin commented Jul 15, 2016

    To reproduce:

    >>> import lzma
    >>> c = lzma.LZMACompressor()
    >>> c.compress(b'')
    b'\xfd7zXZ\x00\x00\x04\xe6\xd6\xb4F'
    >>> c.compress(b'')
    b''
    >>> c.compress(b'')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    _lzma.LZMAError: Insufficient buffer space
    >>> d = lzma.LZMADecompressor()
    >>> d.decompress(b'')
    b''
    >>> d.decompress(b'')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    _lzma.LZMAError: Insufficient buffer space

    This can occur anytime during compression/decompression when an empty string is passed twice in a row.

    The problem is that that liblzma interprets a call to lzma_code() with avail_in == 0 as a buffer full condition. The second time in a row it encounters this, it returns LZMA_BUF_ERROR as per documentation. The attached patch prevents this condition from occurring.

    @benfogle benfogle mannequin added extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error labels Jul 15, 2016
    @AraHaan
    Copy link
    Mannequin

    AraHaan mannequin commented Jul 15, 2016

    Why you passing decompress without passing compress before decompressing it again?

    Also I would expect that it would show the same compress result trice in a row on the 1st test. This must definately be a issue.

    @benfogle
    Copy link
    Mannequin Author

    benfogle mannequin commented Jul 15, 2016

    The above code demonstrates two separate issues. One with the decompressor, and one with the compressor.

    In the compressor example, the first output differs from the rest because it is a file header which is always emitted. That behavior is correct.

    @serhiy-storchaka serhiy-storchaka added the 3.7 (EOL) end of life label Oct 25, 2016
    @serhiy-storchaka serhiy-storchaka self-assigned this Oct 30, 2016
    @serhiy-storchaka
    Copy link
    Member

    Thank you for your report and patch Benjamin. Seems your patch fixes the problem in default case. But the compressor still fails with the raw format.

    >>> import lzma
    >>> FILTERS_RAW_4 = [{"id": lzma.FILTER_DELTA, "dist": 4},
    ...                  {"id": lzma.FILTER_X86, "start_offset": 0x40},
    ...                  {"id": lzma.FILTER_LZMA2, "preset": 4, "lc": 2}]
    >>> c = lzma.LZMACompressor(lzma.FORMAT_RAW, filters=FILTERS_RAW_4)
    >>> c.compress(b'')
    b''
    >>> c.compress(b'')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    _lzma.LZMAError: Insufficient buffer space

    @benfogle
    Copy link
    Mannequin Author

    benfogle mannequin commented Oct 31, 2016

    Ah, thank you. Good catch. I have reworked the patch to handle both cases.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 31, 2016

    New changeset b06f15507978 by Serhiy Storchaka in branch '3.5':
    Issue bpo-27517: LZMA compressor and decompressor no longer raise exceptions if
    https://hg.python.org/cpython/rev/b06f15507978

    New changeset fb64c7a81010 by Serhiy Storchaka in branch '3.6':
    Issue bpo-27517: LZMA compressor and decompressor no longer raise exceptions if
    https://hg.python.org/cpython/rev/fb64c7a81010

    New changeset 98c078fca8e0 by Serhiy Storchaka in branch 'default':
    Issue bpo-27517: LZMA compressor and decompressor no longer raise exceptions if
    https://hg.python.org/cpython/rev/98c078fca8e0

    @serhiy-storchaka
    Copy link
    Member

    Thank you for you contribution Benjamin.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant