classification
Title: Close directory descriptor in scandir iterator on error
Type: resource usage Stage: resolved
Components: Extension Modules Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: benhoyt, gvanrossum, python-dev, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2016-01-14 22:10 by serhiy.storchaka, last changed 2016-02-08 17:35 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
scandir_closedir_on_error.patch serhiy.storchaka, 2016-01-14 22:10 review
Messages (5)
msg258232 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-01-14 22:10
Since scandir iterator has no close method, the most safe way to avoid file descriptors leaks when use os.scandir() is accumulating yielded results into a list before processing them:

    entries = list(os.scandir(path))

But this doesn't save us from all leaks. If an error happened during yielding next entry, opened file descriptor left open.

Proposed patch makes scandir to close the file descriptor not only when an iteration is exhausted, but when any error (expected OSError or MemoryError) is raised.

This is needed to fix possible leaks in os.walk() in 3.5 (see also issue25995).
msg259850 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-02-08 14:41
Could anyone please make a review?
msg259851 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-08 14:48
> Could anyone please make a review?

Done. Enjoy :-)
msg259856 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-02-08 15:57
New changeset 8ec721bb3027 by Serhiy Storchaka in branch '3.5':
Issue #26117: The os.scandir() iterator now closes file descriptor not only
https://hg.python.org/cpython/rev/8ec721bb3027

New changeset ec12fbf449a5 by Serhiy Storchaka in branch 'default':
Issue #26117: The os.scandir() iterator now closes file descriptor not only
https://hg.python.org/cpython/rev/ec12fbf449a5
msg259866 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-08 17:35
Thanks for the fix Serhiy ;-)
History
Date User Action Args
2016-02-08 17:35:41vstinnersetmessages: + msg259866
2016-02-08 15:59:36serhiy.storchakasetstatus: open -> closed
assignee: serhiy.storchaka
resolution: fixed
stage: patch review -> resolved
2016-02-08 15:57:56python-devsetnosy: + python-dev
messages: + msg259856
2016-02-08 14:48:20vstinnersetmessages: + msg259851
2016-02-08 14:41:36serhiy.storchakasetmessages: + msg259850
2016-01-14 22:10:22serhiy.storchakacreate