classification
Title: tarfile fails to close file handles in case of exception
Type: resource usage Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, jcea, lars.gustaebel, python-dev, serhiy.storchaka, ssam, terry.reedy
Priority: normal Keywords: easy, patch

Created on 2012-11-15 12:49 by ssam, last changed 2012-12-15 02:54 by jcea. This issue is now closed.

Files
File name Uploaded Description Edit
tarfile_fd_leaks-2.7.patch serhiy.storchaka, 2012-11-17 20:02 review
tarfile_fd_leaks.patch serhiy.storchaka, 2012-11-17 20:02 review
Messages (8)
msg175616 - (view) Author: Sam Thursfield (ssam) Date: 2012-11-15 12:49
Exceptions such as disk full during extraction cause tarfile to leak file handles. Besides being messy, it causes real problems if, for example, the target file is on a mount that should be unmounted before the program exits - in this case, the unmount will fail as there are still open file handles.

Simplest solution I can see is to change:

    def makefile(self, tarinfo, targetpath):
        """Make a file called targetpath.
        """
        source = self.extractfile(tarinfo)
        target = bltn_open(targetpath, "wb")
        copyfileobj(source, target)
        source.close()
        target.close()

to this:

    def makefile(self, tarinfo, targetpath):
        """Make a file called targetpath.
        """
        source = self.extractfile(tarinfo)
        try:
            with open(targetpath, "wb") as target:
                shutil.copyfileobj(source, target)
        finally:
            source.close()
msg175617 - (view) Author: Sam Thursfield (ssam) Date: 2012-11-15 12:53
sorry, replace 'open' with 'bltn_open' in the above comment - no need to change it.
msg175618 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-15 14:26
Issue16408 is similar issue for zipfile.
msg175714 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-11-16 20:44
Would "with source = self.extractfile(tarinfo):" work?
msg175800 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-17 20:02
> Would "with source = self.extractfile(tarinfo):" work?

No.  extractfile() can return an instance of custom class.  But in 3.x this not 
used at all.

Here are patches for 2.7 and 3.x.
msg176189 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-11-23 17:15
LGTM
msg176641 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-11-29 12:22
New changeset b7bdc0b3c2fe by Andrew Svetlov in branch '3.2':
Issue #16477: Close tarfile internal handlers in case of exception.
http://hg.python.org/cpython/rev/b7bdc0b3c2fe

New changeset 80749ddc30b6 by Andrew Svetlov in branch '3.3':
Merge issue #16477: Close tarfile internal handlers in case of exception.
http://hg.python.org/cpython/rev/80749ddc30b6

New changeset 2d887691a99b by Andrew Svetlov in branch 'default':
Merge issue #16477: Close tarfile internal handlers in case of exception.
http://hg.python.org/cpython/rev/2d887691a99b

New changeset ea4bdf5a2e69 by Andrew Svetlov in branch '2.7':
Issue #16477: Close tarfile internal handlers in case of exception.
http://hg.python.org/cpython/rev/ea4bdf5a2e69
msg176642 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-11-29 12:23
Fixed. Thanks, Serhiy
History
Date User Action Args
2013-03-04 18:14:40ezio.melottilinkissue11787 superseder
2012-12-15 02:54:57jceasetnosy: + jcea
2012-11-29 12:23:37asvetlovsetstatus: open -> closed
resolution: fixed
messages: + msg176642

stage: patch review -> resolved
2012-11-29 12:22:42python-devsetnosy: + python-dev
messages: + msg176641
2012-11-23 17:15:58asvetlovsetnosy: + asvetlov
messages: + msg176189
2012-11-17 23:34:36serhiy.storchakasetstage: needs patch -> patch review
2012-11-17 20:02:13serhiy.storchakasetfiles: + tarfile_fd_leaks-2.7.patch, tarfile_fd_leaks.patch
keywords: + patch
messages: + msg175800
2012-11-16 20:44:16terry.reedysetnosy: + terry.reedy
messages: + msg175714
2012-11-15 14:26:54serhiy.storchakasettype: behavior -> resource usage
versions: + Python 2.7, Python 3.2, Python 3.3, Python 3.4, - Python 3.5
keywords: + easy
nosy: + lars.gustaebel, serhiy.storchaka

messages: + msg175618
stage: needs patch
2012-11-15 12:53:31ssamsetmessages: + msg175617
2012-11-15 12:49:47ssamcreate