classification
Title: python -mzipfile fails to unzip files with folders created by zip
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Antony.Lee, python-dev, ryantimwilson, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2014-08-14 23:27 by Antony.Lee, last changed 2014-08-17 12:28 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
fix_zipfile_empty_dir.patch ryantimwilson, 2014-08-16 19:36 review
Messages (4)
msg225325 - (view) Author: Antony Lee (Antony.Lee) * Date: 2014-08-14 23:27
With Python 3.4.1:

$ mkdir foo; zip -r foo{,}; python -mzipfile -e foo.zip dest
  adding: foo/ (stored 0%)
Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.4/zipfile.py", line 1799, in <module>
    main()
  File "/usr/lib/python3.4/zipfile.py", line 1777, in main
    with open(tgt, 'wb') as fp:
IsADirectoryError: [Errno 21] Is a directory: 'dest/foo/'

Note that zipfile is actually able to unzip the file:

$ python -c 'from zipfile import *; ZipFile("foo.zip").extractall("dest")'

works fine.

If the zip file is created by "python -mzipfile -c foo.zip foo" then there is no problem.  Likewise, if there are no folders in the zip file (but just a collection of files) then there is no problem.
msg225410 - (view) Author: Ryan Wilson (ryantimwilson) * Date: 2014-08-16 19:36
The reason behind this was that zipfile.py currently handles empty directories in zipfiles incorrectly.

On lines 1774 - 1778 in Lib/zipfile.py:

tgtdir = os.path.dirname(tgt)
if not os.path.exists(tgtdir):
    os.makedirs(tgtdir)
with open(tgt, 'wb') as fp:
    fp.write(zf.read(path))

In the case described above, tgt is 'dest/foo/' because the directory is empty. For non-empty directories, tgt would be a file in the directory i.e. 'dest/foo/a'. In the empty directory case, the directory will be created, but then opened as file (which will throw the error shown above).

When compressing the file with 'python -mzipfile -c', zipfile.py would not add empty directories to the zipfile. Hence, the zip file generated is empty.

This patch fixes both issues. In the decompression case, I utilize the Zipfile.extractall() function instead of extracting each file manually. The extractall() function handles empty directories correctly. For the compression case, I added a check to add an empty directory to the zip file.
msg225438 - (view) Author: Roundup Robot (python-dev) Date: 2014-08-17 12:20
New changeset ffa5bfe75c3a by Serhiy Storchaka in branch '2.7':
Issue #22201: Command-line interface of the zipfile module now correctly
http://hg.python.org/cpython/rev/ffa5bfe75c3a

New changeset 7b933005c492 by Serhiy Storchaka in branch '3.4':
Issue #22201: Command-line interface of the zipfile module now correctly
http://hg.python.org/cpython/rev/7b933005c492

New changeset dc77ad3a17aa by Serhiy Storchaka in branch 'default':
Issue #22201: Command-line interface of the zipfile module now correctly
http://hg.python.org/cpython/rev/dc77ad3a17aa
msg225439 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-17 12:28
Applied first part of the patch. Thank you Ryan. For the rest please open separate issue.
History
Date User Action Args
2014-08-17 12:28:33serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg225439

stage: patch review -> resolved
2014-08-17 12:20:25python-devsetnosy: + python-dev
messages: + msg225438
2014-08-16 21:22:52serhiy.storchakasetversions: + Python 2.7, Python 3.5
nosy: + serhiy.storchaka

assignee: serhiy.storchaka
type: behavior
stage: patch review
2014-08-16 19:36:32ryantimwilsonsetfiles: + fix_zipfile_empty_dir.patch
versions: - Python 3.5
nosy: + ryantimwilson

messages: + msg225410

keywords: + patch
2014-08-14 23:27:42Antony.Leecreate