classification
Title: ZipFile.read() should mention that it might throw NotImplementedError
Type: enhancement Stage: needs patch
Components: Documentation Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ezio.melotti Nosy List: Tuikku.Anttila, detly, docs@python, ezio.melotti, gregory.p.smith, loewis, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2014-07-23 12:20 by detly, last changed 2015-04-14 17:07 by gregory.p.smith. This issue is now closed.

Files
File name Uploaded Description Edit
issue22046.patch Tuikku.Anttila, 2014-08-02 12:12 review
22046_1.patch Tuikku.Anttila, 2014-08-05 18:06 review
Scheme.zip detly, 2014-08-06 03:40 Archive containing other archives with different compression methods
zftest.py detly, 2014-08-06 03:40 Script to run on "Scheme.zip" to demonstrate the problem
Messages (12)
msg223737 - (view) Author: Jason Heeris (detly) Date: 2014-07-23 12:20
As per issue 5701, the zipfile.ZipFile.read() method will throw a NotImplementedError if the compression scheme is not supported. However, there is no mention of this possibility in the documentation for the read() method. I would suggest, say, "Calling read() on a ZipFile that uses an unsupported compression scheme (eg. implode) will raise a NotImplementedError."

It looks like you can use the testzip() method to check that this won't happen (ie. after you open the file but before you extract an entry). If that is really the expected way to check for this kind of condition, it would be nice to mention that too (under either method).
msg224556 - (view) Author: Tuikku Anttila (Tuikku.Anttila) * Date: 2014-08-02 12:12
Added to the documentation of zipfile.ZipFile.read() that the method will throw a NotImplementedError when the compression scheme of the ZipFile is something else than ZIP_STORED, ZIP_DEFLATED, ZIP_BZIP2 or ZIP_LZMA.
msg224868 - (view) Author: Tuikku Anttila (Tuikku.Anttila) * Date: 2014-08-05 18:06
Added mention that an error might also be raised if the method is not available.
msg224897 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-08-06 03:11
After further investigation it seems to me that read can't raise NotImplementedError.  ZipFile also won't raise it, but will raise a RuntimeError:
>>> zipfile.ZipFile('spam.zip', 'w', compression=zipfile.ZIP_BZIP2)
RuntimeError: Compression requires the (missing) bz2 module

By looking at the code, ZipFile calls _check_compression (Lib/zipfile.py:904) and _check_compression raises RuntimeError (Lib/zipfile.py:579).
ZipExtFile calls _get_decompressor (Lib/zipfile.py:651) and _get_decompressor raises NotImplemented (Lib/zipfile.py:610).
This behavior seems inconsistent and perhaps should be fixed (in this case a new issue should be created), however this will probably be backward-incompatible.

Regardless of this, it seems that currently NotImplementedError can be raised in some situations, and the zipfile docs don't mention it, so the doc can still be improved.

@Martin
Do you have any opinion on the aforementioned inconsistency?

@Jason
Did you actually manage to get a NotImplementedError from ZipFile.read() or from somewhere else?
msg224898 - (view) Author: Jason Heeris (detly) Date: 2014-08-06 03:14
@Ezio

I am pretty sure it was read(). I couldn't submit the file I used as an example, so I'll see if I can construct a minimal example to post here.
msg224899 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-08-06 03:25
FWIW #5701 has a test zipfile (I haven't tried it though).

If I'm reading the code right, the compression method is specified and checked in the __init__, so we should know if the compression method was available long before we reach .read().  I will be happy to see some tests that prove me wrong (or confirm what I said), so that we know how the documentation should be updated.
msg224900 - (view) Author: Jason Heeris (detly) Date: 2014-08-06 03:41
Okay, I've attached two files:

1. Scheme.zip, from issue 5701
2. "zftest.py", a script that you run in the same dir as "Scheme.zip" to produce this:

$ python zftest.py 
Extracting: 1!SCHEME.Z64
Traceback (most recent call last):
  File "zftest.py", line 8, in <module>
    child_data = parent.read(zinfo)
  File "/usr/lib/python2.7/zipfile.py", line 931, in read
    return self.open(name, "r", pwd).read()
  File "/usr/lib/python2.7/zipfile.py", line 1006, in open
    close_fileobj=should_close)
  File "/usr/lib/python2.7/zipfile.py", line 530, in __init__
    raise NotImplementedError("compression type %d (%s)" % (self._compress_type, descr))
NotImplementedError: compression type 6 (implode)
msg224901 - (view) Author: Jason Heeris (detly) Date: 2014-08-06 03:42
Sorry, that was run with Python 2.7.5+ on Ubuntu.
msg224912 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-06 06:53
ZipFile's constructor and ZipFile.open() can raise NotImplementedError, RuntimeError, EOFError, IOError, OSError or its subclasses, or any exception raised from underlying file object, including TypeError and AttributeError. ZipExtFile.read() can raise zlib.error, lzma.LZMAError, EOFError, IOError, OSError, etc. Any method can raise unexpected exception when used in unusual circumstances (with threads, in signal handler, in destructor, at shutdown stage). I don't think we should document all these exception. Python documentation never document all possible exceptions raised by a method.
msg224913 - (view) Author: Jason Heeris (detly) Date: 2014-08-06 07:01
> Python documentation never document all possible exceptions raised by a method.

No, but clearly *some* exceptions are documented, and presumably there's some reasoning behind which are and aren't.

In this case, the NotImplemented error is there by design. It's not an incidental effect of something else. It's part of the API, and it's used to indicate a common error condition: that the compression format is not supported.
msg240955 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-14 17:05
New changeset 3e8047ee9bbb by Gregory P. Smith in branch '3.4':
issue22046: mention that zipfile can raise NotImplementedError on unsupported
https://hg.python.org/cpython/rev/3e8047ee9bbb

New changeset 4b9deb7e6f2b by Gregory P. Smith in branch 'default':
issue22046: mention that zipfile can raise NotImplementedError on unsupported
https://hg.python.org/cpython/rev/4b9deb7e6f2b
msg240956 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2015-04-14 17:06
fyi - i didn't update the 2.7 docs. just 3.4 and 3.5. if some committer wants to, feel free.
History
Date User Action Args
2015-04-14 17:07:06gregory.p.smithsetstatus: open -> closed
resolution: fixed
2015-04-14 17:06:57gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg240956
2015-04-14 17:05:19python-devsetnosy: + python-dev
messages: + msg240955
2014-08-06 07:01:31detlysetmessages: + msg224913
2014-08-06 06:53:23serhiy.storchakasetmessages: + msg224912
2014-08-06 03:42:14detlysetmessages: + msg224901
2014-08-06 03:41:26detlysetmessages: + msg224900
2014-08-06 03:40:41detlysetfiles: + zftest.py
2014-08-06 03:40:16detlysetfiles: + Scheme.zip
2014-08-06 03:25:18ezio.melottisetmessages: + msg224899
2014-08-06 03:14:18detlysetmessages: + msg224898
2014-08-06 03:11:46ezio.melottisetassignee: docs@python -> ezio.melotti

messages: + msg224897
nosy: + serhiy.storchaka, ezio.melotti, loewis
2014-08-05 18:06:48Tuikku.Anttilasetfiles: + 22046_1.patch

messages: + msg224868
2014-08-02 12:12:41Tuikku.Anttilasetfiles: + issue22046.patch

nosy: + Tuikku.Anttila
messages: + msg224556

keywords: + patch
2014-08-01 06:30:38serhiy.storchakasetstage: needs patch
versions: + Python 3.4, Python 3.5, - Python 3.3
2014-07-23 12:20:23detlycreate