classification
Title: decompressing and then re-compressing zipfiles with Python 3 zipfile loses flag_bits
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: keeely, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2018-11-12 14:38 by keeely, last changed 2019-03-12 16:01 by keeely.

Files
File name Uploaded Description Edit
zipfile_flags.patch keeely, 2018-11-13 11:24
tmp.py keeely, 2019-03-12 15:46
Messages (6)
msg329733 - (view) Author: keeely (keeely) Date: 2018-11-12 14:38
Python 2.7 allows one to capture from one file a list of ZipInfo objects, and then recreate a new file using those ZipInfo objects.  The same thing appears to be impossible with Python 3 without resorting to monkey-patches because of a line of code in the writestr() code path ignoring the passed in value of ZipInfo and overwritting it with zeros.

See:
https://github.com/python/cpython/blob/master/Lib/zipfile.py#L1567

See also a possible solution at:
https://stackoverflow.com/questions/53254622/zipfile-header-language-encoding-bit-set-differently-between-python2-and-python3
msg332929 - (view) Author: keeely (keeely) Date: 2019-01-03 14:29
Please note:  I'm unable to fill in your contributor agreement form, so please consider the patch an illustrative example.  In any case, the fix is pretty-much a one-liner, so shouldn't be a big deal to 're-write'.

I'm disappointed that this has been largely overlooked since I raised it.  It's a clear regression in Python3 and represents a show-stopper for people wanting to switch from 2->3.  I would imagine you *want* people to migrate to 3, so I'm baffled by the response or lack thereof.  Sure, I know everyone is busy but this is pretty basic stuff.
msg333171 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-01-07 15:43
Hi,

> I'm unable to fill in your contributor agreement form

Is there a specific reason why you cannot fill it?

If you cannot cannot fill it, can someone else convert your patch into a PR?
msg337761 - (view) Author: keeely (keeely) Date: 2019-03-12 15:46
Can you please get on and fix this if you're going to deprecate Python 2.0.  It's unkind on your users to deprecate the old version before you've fixed basic regressions like this in the new one.  Just to be clear, I am attaching the code that reproduces this (tmp.py).   Run it with Python 2 and 3 to see the difference.  I will not fill in your agreement form, and no I will not provide my reasons, but you don't need it, it's a one-line fix.
msg337764 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-12 15:55
> I will not fill in your agreement form, and no I will not provide my reasons, but you don't need it, it's a one-line fix.

Ok. It's up to you, but in that case, we cannot merge your change. Contributors have to sign it. Your test is larger than a single line.
msg337768 - (view) Author: keeely (keeely) Date: 2019-03-12 16:01
You can take out the test.  It wasn't there before so who's going to care?
History
Date User Action Args
2019-03-12 16:01:43keeelysetmessages: + msg337768
2019-03-12 15:55:09vstinnersetmessages: + msg337764
2019-03-12 15:46:18keeelysetfiles: + tmp.py

messages: + msg337761
2019-01-07 15:43:13vstinnersetnosy: + vstinner
messages: + msg333171
2019-01-03 14:59:17xtreaksetnosy: + serhiy.storchaka
2019-01-03 14:29:18keeelysetmessages: + msg332929
2018-11-13 11:24:00keeelysetfiles: + zipfile_flags.patch
keywords: + patch
2018-11-12 21:23:34keeelysettitle: Impossible to round-trip decompress -> compress zipfiles with Python 3 due to losing flag_bits -> decompressing and then re-compressing zipfiles with Python 3 zipfile loses flag_bits
2018-11-12 14:38:21keeelycreate