This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ZipFileExt.read() reads more data than requested
Type: Stage:
Components: Library (Lib) Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Black.Dew, pitrou
Priority: normal Keywords:

Created on 2010-09-12 09:28 by Black.Dew, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
a.py Black.Dew, 2010-09-12 14:03 simple test
Messages (4)
msg116158 - (view) Author: Black Dew (Black.Dew) Date: 2010-09-12 09:28
ZipFileExt.read() can return more data than requested, unlike file and other file-like objects.

This function calls read1() in a loop, passing the original requested size even if part of the data was already read thus reading and returning more than the caller requested.

This should be fixed by requesting n-len(buf) or something similar, or at least properly documented.
msg116184 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-12 12:44
Indeed. In most cases, though, read1(n) will return exactly n bytes and the loop will therefore end immediately with the right buf size. Do you have a simple test case?
msg116197 - (view) Author: Black Dew (Black.Dew) Date: 2010-09-12 14:03
If i follow the logic in read1() correctly it will do that only for files with very low compression ratios (the original sample where i noticed that problem was actually a chunk of encrypted data inside the zip).

From the comments referring to "with at most one read() system call" I would also assume it's OS-specific.

Anyway, I've attached a small file that reproduces this behavior on my system (win xp 32bit). When i run it i get "AssertionError: Read returned 4097 bytes (4096 requested)"
msg116203 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-12 14:58
Ok, thank you. I've fixed the issue and added a test in r84737 (3.x) and r84739 (2.7).
History
Date User Action Args
2022-04-11 14:57:06adminsetgithub: 54046
2010-09-12 14:58:36pitrousetstatus: open -> closed
resolution: fixed
messages: + msg116203
2010-09-12 14:03:34Black.Dewsetfiles: + a.py

messages: + msg116197
2010-09-12 12:44:39pitrousetmessages: + msg116184
versions: + Python 3.2
2010-09-12 09:38:39r.david.murraysetnosy: + pitrou
2010-09-12 09:28:13Black.Dewcreate