Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Close directory descriptor in scandir iterator on error #70305

Closed
serhiy-storchaka opened this issue Jan 14, 2016 · 5 comments
Closed

Close directory descriptor in scandir iterator on error #70305

serhiy-storchaka opened this issue Jan 14, 2016 · 5 comments
Assignees
@serhiy-storchaka
Labels
extension-modules C modules in the Modules dir performance Performance or resource usage

Comments

@serhiy-storchaka
Copy link
Member

BPO 26117
Nosy @gvanrossum, @vstinner, @benhoyt, @serhiy-storchaka
Files
  • scandir_closedir_on_error.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-02-08.15:59:36.282>
created_at = <Date 2016-01-14.22:10:22.722>
labels = ['extension-modules', 'performance']
title = 'Close directory descriptor in scandir iterator on error'
updated_at = <Date 2016-02-08.17:35:41.304>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2016-02-08.17:35:41.304>
actor = 'vstinner'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-02-08.15:59:36.282>
closer = 'serhiy.storchaka'
components = ['Extension Modules']
creation = <Date 2016-01-14.22:10:22.722>
creator = 'serhiy.storchaka'
dependencies = []
files = ['41616']
hgrepos = []
issue_num = 26117
keywords = ['patch']
message_count = 5.0
messages = ['258232', '259850', '259851', '259856', '259866']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'vstinner', 'benhoyt', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue26117'
versions = ['Python 3.5', 'Python 3.6']
    @serhiy-storchaka
    Copy link
    Member Author

    Since scandir iterator has no close method, the most safe way to avoid file descriptors leaks when use os.scandir() is accumulating yielded results into a list before processing them:

        entries = list(os.scandir(path))

    But this doesn't save us from all leaks. If an error happened during yielding next entry, opened file descriptor left open.

    Proposed patch makes scandir to close the file descriptor not only when an iteration is exhausted, but when any error (expected OSError or MemoryError) is raised.

    This is needed to fix possible leaks in os.walk() in 3.5 (see also bpo-25995).

    @serhiy-storchaka serhiy-storchaka added extension-modules C modules in the Modules dir performance Performance or resource usage labels Jan 14, 2016
    @serhiy-storchaka
    Copy link
    Member Author

    Could anyone please make a review?

    @vstinner
    Copy link
    Member

    vstinner commented Feb 8, 2016

    Could anyone please make a review?

    Done. Enjoy :-)

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Feb 8, 2016

    New changeset 8ec721bb3027 by Serhiy Storchaka in branch '3.5':
    Issue bpo-26117: The os.scandir() iterator now closes file descriptor not only
    https://hg.python.org/cpython/rev/8ec721bb3027

    New changeset ec12fbf449a5 by Serhiy Storchaka in branch 'default':
    Issue bpo-26117: The os.scandir() iterator now closes file descriptor not only
    https://hg.python.org/cpython/rev/ec12fbf449a5

    @serhiy-storchaka serhiy-storchaka self-assigned this Feb 8, 2016
    @vstinner
    Copy link
    Member

    vstinner commented Feb 8, 2016

    Thanks for the fix Serhiy ;-)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @serhiy-storchaka serhiy-storchaka

    Labels
    extension-modules C modules in the Modules dir performance Performance or resource usage
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @vstinner @serhiy-storchaka