classification
Title: FD leaks in zipfile
Type: resource usage Stage: resolved
Components: Library (Lib) Versions: Python 3.4, Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Eric.Busboom, jcea, pitrou, python-dev, serhiy.storchaka
Priority: normal Keywords: easy, patch

Created on 2012-11-04 20:22 by Eric.Busboom, last changed 2012-11-17 22:59 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
zipfile_fd_leaks.patch serhiy.storchaka, 2012-11-05 21:26 Patch for 3.3 and 3.4 review
zipfile_fd_leaks-3.2.patch serhiy.storchaka, 2012-11-05 21:27 Patch for 3.2 review
zipfile_fd_leaks-2.7.patch serhiy.storchaka, 2012-11-05 21:28 Patch for 2.7 review
Messages (10)
msg174829 - (view) Author: Eric Busboom (Eric.Busboom) Date: 2012-11-04 20:22
The zipfile.testzip() method will open each of the files in a zip archive, but does not close the files, resulting in a file descriptor leak.
msg174837 - (view) Author: Eric Busboom (Eric.Busboom) Date: 2012-11-04 22:53
I've tried just closing the ZipExtFile created in testzip, but that didn't actually close the file. It looks like ZipExtClose.close() also doesn't close the file descriptor, at least when the ZipFile is constructed on a filename.

This code worked (Addition of f._fileobj.close() )

f = zf.open(zinfo.filename, "r")
while f.read( 2 ** 20):     # Check CRC-32
  pass
f.close()
f._fileobj.close() # This shoulnd't be necessary, but it is.
msg174913 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-05 14:57
Quite the contrary.  The ZipExtFile closes the file descriptor if the ZipFile is constructed on a filename.
msg174921 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2012-11-05 17:22
I think this is a duplicate of issue #16183. Are you sure python 3 is affected?.
msg174923 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-05 17:36
No, this is not a duplicate.  ZipExtFile.close() closes a file handle, but ZipExtFile.close() is not used here.  I see also some possible FD leaks in ZipFile.open().  It will be good to fix all them in this issue.  If no one will make a patch for this easy issue, I'll do it.
msg174936 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2012-11-05 19:51
Serhiy Storchaka, go for it.
msg174942 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-05 21:26
Here is a patch.  In additional to leaks in testzip and ZipFile.open() I fixed possible leaks in other places.
msg175824 - (view) Author: Roundup Robot (python-dev) Date: 2012-11-17 22:56
New changeset d51d95c9ea81 by Antoine Pitrou in branch '3.2':
Issue #16408: Fix file descriptors not being closed in error conditions in the zipfile module.
http://hg.python.org/cpython/rev/d51d95c9ea81

New changeset 27cb1a3d57c8 by Antoine Pitrou in branch '3.3':
Issue #16408: Fix file descriptors not being closed in error conditions in the zipfile module.
http://hg.python.org/cpython/rev/27cb1a3d57c8

New changeset 779e8f31dd30 by Antoine Pitrou in branch 'default':
Issue #16408: Fix file descriptors not being closed in error conditions in the zipfile module.
http://hg.python.org/cpython/rev/779e8f31dd30
msg175825 - (view) Author: Roundup Robot (python-dev) Date: 2012-11-17 22:58
New changeset eba28506eed3 by Antoine Pitrou in branch '2.7':
Issue #16408: Fix file descriptors not being closed in error conditions in the zipfile module.
http://hg.python.org/cpython/rev/eba28506eed3
msg175826 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-11-17 22:59
Thank you for the patches. They looked fine, so I committed them.
History
Date User Action Args
2012-11-17 22:59:16pitrousetstatus: open -> closed

nosy: + pitrou
messages: + msg175826

resolution: fixed
stage: patch review -> resolved
2012-11-17 22:58:43python-devsetmessages: + msg175825
2012-11-17 22:56:09python-devsetnosy: + python-dev
messages: + msg175824
2012-11-05 21:28:02serhiy.storchakasetfiles: + zipfile_fd_leaks-2.7.patch
2012-11-05 21:27:15serhiy.storchakasetfiles: + zipfile_fd_leaks-3.2.patch
2012-11-05 21:26:33serhiy.storchakasetfiles: + zipfile_fd_leaks.patch
title: zipfile.testzip() opens file but does not close it. -> FD leaks in zipfile
messages: + msg174942

keywords: + patch
stage: needs patch -> patch review
2012-11-05 19:51:52jceasetmessages: + msg174936
2012-11-05 17:36:52serhiy.storchakasetmessages: + msg174923
2012-11-05 17:22:18jceasetnosy: + jcea
messages: + msg174921
2012-11-05 14:57:39serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg174913
2012-11-04 22:53:22Eric.Busboomsetmessages: + msg174837
2012-11-04 22:13:05serhiy.storchakasetkeywords: + easy
stage: needs patch
type: behavior -> resource usage
versions: + Python 3.2, Python 3.3, Python 3.4
2012-11-04 20:22:59Eric.Busboomcreate