classification
Title: Zipfile breaks if signalled during write()
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alanmcintyre, and800, serhiy.storchaka, twouters
Priority: normal Keywords: patch

Created on 2019-03-26 08:59 by and800, last changed 2019-03-30 13:54 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12559 merged serhiy.storchaka, 2019-03-26 12:25
PR 12628 merged miss-islington, 2019-03-30 06:25
Messages (4)
msg338863 - (view) Author: Andriy Maletsky (and800) * Date: 2019-03-26 08:59
Consider a simple write to a zip file:


import zipfile

with zipfile.ZipFile('/workdir/archive.zip', 'w', compression=zipfile.ZIP_DEFLATED) as zip_archive:
    zip_archive.write('/workdir/data.csv', arcname='data.csv')
    print('exiting from context manager...')


If a signal handler is fired and raises an exception during certain points of write() execution, such an error occurs (py 3.7.2):


Traceback (most recent call last):
  File "zipissue.py", line 4, in <module>
    zip_archive.write('/workdir/data.csv', arcname='data.csv')
  File "/usr/local/lib/python3.7/zipfile.py", line 1744, in write
    shutil.copyfileobj(src, dest, 1024*8)
  File "/usr/local/lib/python3.7/zipfile.py", line 1107, in close
    buf = self._compressor.flush()
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "zipissue.py", line 5, in <module>
    print('exiting from context manager...')
  File "/usr/local/lib/python3.7/zipfile.py", line 1265, in __exit__
    self.close()
  File "/usr/local/lib/python3.7/zipfile.py", line 1798, in close
    raise ValueError("Can't close the ZIP file while there is "
ValueError: Can't close the ZIP file while there is an open writing handle on it. Close the writing handle before closing the zip.
Exception ignored in: <function ZipFile.__del__ at 0x7fbeea4fcd08>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/zipfile.py", line 1789, in __del__
  File "/usr/local/lib/python3.7/zipfile.py", line 1798, in close
ValueError: Can't close the ZIP file while there is an open writing handle on it. Close the writing handle before closing the zip.


Before any write the `ZipFile._writing` flag is set, and that flag is cleared at `_ZipWriteFile.close()`. But if signalled inside `_ZipWriteFile.close()` we are moving to a broken state: we don't write anything anymore, but `ZipFile._writing` is still set. Therefore we cannot clearly close ZipFile. As ZipFile contextmanager swallows KeyboardInterrupt and produces an exception of `Exception` type, this leads to the impossibility of proper program shutdown.

I believe that by simply moving `ZipFile._writing = False` in `_ZipWriteFile.close()` to some finally block, this issue will be solved safely.
msg339178 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-03-30 06:25
New changeset 2524fdefc9bb2a97b99319190aeb23703079ad4c by Serhiy Storchaka in branch 'master':
bpo-36434: Properly handle writing errors in ZIP files. (GH-12559)
https://github.com/python/cpython/commit/2524fdefc9bb2a97b99319190aeb23703079ad4c
msg339198 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-03-30 13:52
New changeset 4724ba9b57c45ce4bca3c828f2ed8dcff3800a0c by Serhiy Storchaka (Miss Islington (bot)) in branch '3.7':
bpo-36434: Properly handle writing errors in ZIP files. (GH-12559) (GH-12628)
https://github.com/python/cpython/commit/4724ba9b57c45ce4bca3c828f2ed8dcff3800a0c
msg339200 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-03-30 13:54
Thank you for your report Andriy. 3.5 and 3.6 take only security bug fixes.
History
Date User Action Args
2019-03-30 13:54:25serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-03-30 13:54:11serhiy.storchakasetmessages: + msg339200
versions: - Python 3.5, Python 3.6, Python 3.9
2019-03-30 13:52:21serhiy.storchakasetmessages: + msg339198
2019-03-30 06:25:29miss-islingtonsetpull_requests: + pull_request12561
2019-03-30 06:25:22serhiy.storchakasetmessages: + msg339178
2019-03-26 12:25:08serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request12504
2019-03-26 09:08:03xtreaksetnosy: + twouters, alanmcintyre, serhiy.storchaka
2019-03-26 08:59:02and800create