Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compression level for zipfile #65616

Closed
Sworddragon mannequin opened this issue May 2, 2014 · 12 comments
Closed

Compression level for zipfile #65616

Sworddragon mannequin opened this issue May 2, 2014 · 12 comments
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@Sworddragon
Copy link
Mannequin

Sworddragon mannequin commented May 2, 2014

BPO 21417
Nosy @gpshead, @berkerpeksag, @serhiy-storchaka, @AraHaan, @bbayles
PRs
  • bpo-21417: Compression level for zipfile #5385
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-01-30.06:52:14.039>
created_at = <Date 2014-05-02.18:00:16.328>
labels = ['3.7', 'type-feature', 'library']
title = 'Compression level for zipfile'
updated_at = <Date 2018-02-26.22:50:36.558>
user = 'https://bugs.python.org/Sworddragon'

    bugs.python.org fields:

    activity = <Date 2018-02-26.22:50:36.558>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2018-01-30.06:52:14.039>
closer = 'gregory.p.smith'
components = ['Library (Lib)']
creation = <Date 2014-05-02.18:00:16.328>
creator = 'Sworddragon'
dependencies = []
files = []
hgrepos = []
issue_num = 21417
keywords = ['patch']
message_count = 10.0
messages = ['217770', '245753', '272610', '272611', '272646', '311237', '312883', '312921', '312961', '312968']
nosy_count = 10.0
nosy_names = ['gregory.p.smith', 'alanmcintyre', 'Sworddragon', 'fberger', 'berker.peksag', 'jens', 'serhiy.storchaka', 'jfoo', 'Decorater', 'bbayles']
pr_nums = ['5385']
priority = 'normal'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21417'
versions = ['Python 3.7']
    @Sworddragon
    Copy link
    Mannequin Author

    Sworddragon mannequin commented May 2, 2014

    This is a fork from this ticket: http://bugs.python.org/issue21404

    tarfile has a compression level and seems to get now the missing documentation for it. But there is still a compression level missing for zipfile.

    @Sworddragon Sworddragon mannequin added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels May 2, 2014
    @jens
    Copy link
    Mannequin

    jens mannequin commented Jun 24, 2015

    btw. hacked work-a-round is:

        zlib.Z_DEFAULT_COMPRESSION = 9

    @jfoo
    Copy link
    Mannequin

    jfoo mannequin commented Aug 13, 2016

    Would compresslevel be a suitable argument? All the backend compressors accept values from 1 to 9 if I'm not mistaken.

    @AraHaan
    Copy link
    Mannequin

    AraHaan mannequin commented Aug 13, 2016

    Yes, it would make sence or better yet have a arg that is optional that if not set make it = 9.

    @jfoo
    Copy link
    Mannequin

    jfoo mannequin commented Aug 14, 2016

    I thought about that, but:

    1. compresslevel=None might make look like no compression was being done at all
    2. The current default is not 9 for all backends

    @gpshead
    Copy link
    Member

    gpshead commented Jan 30, 2018

    New changeset ce237c7 by Gregory P. Smith (Bo Bayles) in branch 'master':
    bpo-21417: Add compresslevel= to the zipfile module (GH-5385)
    ce237c7

    @gpshead gpshead added the 3.7 (EOL) end of life label Jan 30, 2018
    @gpshead gpshead closed this as completed Jan 30, 2018
    @serhiy-storchaka
    Copy link
    Member

    I have a concern about the _compresslevel attribute of ZipInfo. Why is it underscored and not documented?

    @bbayles
    Copy link
    Mannequin

    bbayles mannequin commented Feb 26, 2018

    I made a comment about that in the pull request that went unchallenged ([1]), but I'm happy to change it.

    [1] #5385 (review)

    @gpshead
    Copy link
    Member

    gpshead commented Feb 26, 2018

    keeping the conversation in one place, the code review comment was: "I've prefixed this with an underscore because it's not exposed when reading back. ZipInfo has another "private" attribute, _raw_time"

    compresslevel is documented on the public APIs where it appears, I do not believe it needs to be a public attribute within ZipInfo given the above as that is an implementation detail. If we ever find a good reason to make it public in the future we can add a property.

    @serhiy-storchaka
    Copy link
    Member

    This makes sense to me. Thank you for explanation.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @zborro
    Copy link

    zborro commented Nov 4, 2022

    @gpshead so is it a bug or a feature that adding files to archive using ZipInfo completely ignores what compression type and compression level is set in the ZipFile instance?

    In the library code there is:

            # Make sure we have an info object
        if isinstance(name, ZipInfo):
            # 'name' is already an info object
            zinfo = name

    So it takes ZipInfo as is. If one wants to set compression level, that private field needs to be set:

    zipinfo._compresslevel = 9

    Or am I missing something here?

    lazka added a commit to lazka/meson-python that referenced this issue Oct 21, 2023
    @lazka
    Fix missing compression of wheels
    aac6839 
    As pointed out in https://docs.python.org/3/library/zipfile.html#zipfile.ZipFile.writestr
when creating a ZipInfo it defaults to ZIP_STORED, which means the compression
passed to zipfile.ZipFile() is ignored. This leads to the wheels files being
uncompressed.

To fix this copy over the compression type from the ZipFile instance.
Note that the same is true for the compression level, but it's not used anyway,
and there is currently no public API to set it, see
python/cpython#65616 (comment)

Fixes mesonbuild#519
    lazka added a commit to lazka/meson-python that referenced this issue Oct 21, 2023
    @lazka
    Fix wheels not being compressed
    3198506 
    As pointed out in https://docs.python.org/3/library/zipfile.html#zipfile.ZipFile.writestr
when creating a ZipInfo it defaults to ZIP_STORED, which means the compression
passed to zipfile.ZipFile() is ignored. This leads to the wheels files being
uncompressed.

To fix this copy over the compression type from the ZipFile instance.
Note that the same is true for the compression level, but it's not used,
and there is currently no public API to set it, see
python/cpython#65616 (comment)

Fixes mesonbuild#519
    @lazka lazka mentioned this issue Oct 21, 2023
    Merged
    lazka added a commit to lazka/meson-python that referenced this issue Oct 21, 2023
    @lazka
    Fix wheels not being compressed
    ddc658b 
    As pointed out in https://docs.python.org/3/library/zipfile.html#zipfile.ZipFile.writestr
when creating a ZipInfo it defaults to ZIP_STORED, which means the compression
passed to zipfile.ZipFile() is ignored. This leads to the wheel file being
uncompressed.

To fix this copy over the compression type from the ZipFile instance.
Note that the same is true for the compression level, but it's not used,
and there is currently no public API to set it, see
python/cpython#65616 (comment)

Fixes mesonbuild#519
    lazka added a commit to lazka/meson-python that referenced this issue Oct 21, 2023
    @lazka
    BUG: Fix wheels not being compressed
    ee7d9e4 
    As pointed out in https://docs.python.org/3/library/zipfile.html#zipfile.ZipFile.writestr
when creating a ZipInfo it defaults to ZIP_STORED, which means the compression
passed to zipfile.ZipFile() is ignored. This leads to the wheel file being
uncompressed.

To fix this copy over the compression type from the ZipFile instance.
Note that the same is true for the compression level, but it's not used,
and there is currently no public API to set it, see
python/cpython#65616 (comment)

Fixes mesonbuild#519
    @lazka
    Copy link
    Contributor

    lazka commented Oct 21, 2023 •
    edited

    So it takes ZipInfo as is. If one wants to set compression level, that private field needs to be set:

    zipinfo._compresslevel = 9

    Or am I missing something here?

    @zborro Because I just had a similar issue. You don't need to set the private field, you can pass compresslevel to writestr() instead: zip_file.writestr(zip_info, compress_type=ZIP_DEFLATED, compresslevel=9)

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @gpshead @lazka @serhiy-storchaka @zborro