Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

shutil.rmtree() FileNotFoundError race condition #81441

Closed
websurfer5 mannequin opened this issue Jun 13, 2019 · 3 comments
Closed

shutil.rmtree() FileNotFoundError race condition #81441

websurfer5 mannequin opened this issue Jun 13, 2019 · 3 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir

Comments

@websurfer5
Copy link
Mannequin

websurfer5 mannequin commented Jun 13, 2019

BPO 37260
Nosy @giampaolo, @tarekziade, @websurfer5, @iritkatriel
PRs
  • gh-81441: shutil.rmtree() FileNotFoundError race condition #14064
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2019-06-13.01:10:41.338>
labels = ['3.8', '3.7', 'library', '3.9']
title = 'shutil.rmtree() FileNotFoundError race condition'
updated_at = <Date 2021-06-20.14:54:37.725>
user = 'https://github.com/websurfer5'

    bugs.python.org fields:

    activity = <Date 2021-06-20.14:54:37.725>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2019-06-13.01:10:41.338>
creator = 'Jeffrey.Kintscher'
dependencies = []
files = []
hgrepos = []
issue_num = 37260
keywords = ['patch']
message_count = 3.0
messages = ['345445', '345719', '396176']
nosy_count = 4.0
nosy_names = ['giampaolo.rodola', 'tarek', 'Jeffrey.Kintscher', 'iritkatriel']
pr_nums = ['14064']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue37260'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']
    @websurfer5
    Copy link
    Mannequin Author

    websurfer5 mannequin commented Jun 13, 2019

    shutil.rmtree() is susceptible to a race condition that can needlessly raise OSError:

    1. os.scandir() returns the list of entries in a directory
    2. while iterating over the list, another thread or process deletes one or more of the entries not yet iterated
    3. os.unlink() or stat() raises OSError for the already deleted entry

    It should check for and ignore, at a minimum, FileNotFoundError because we were going to delete the entry anyways. For comparison, the 'rm -r' shell command handles this race condition by ignoring entries deleted from under it.

    I will submit a PR when I work out some test cases to include.

    @websurfer5 websurfer5 mannequin added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir labels Jun 13, 2019
    @websurfer5
    Copy link
    Mannequin Author

    websurfer5 mannequin commented Jun 16, 2019

    The PR is ready for review.

    @iritkatriel
    Copy link
    Member

    see also bpo-29699

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @bedevere-bot bedevere-bot mentioned this issue Feb 9, 2023
    Merged
    serhiy-storchaka added a commit that referenced this issue Dec 5, 2023
    @websurfer5 @serhiy-storchaka
    gh-81441: shutil.rmtree() FileNotFoundError race condition (GH-14064)
    268415b 
    Ignore missing files and directories while enumerating
directory entries in shutil.rmtree().

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
    @serhiy-storchaka serhiy-storchaka mentioned this issue Dec 5, 2023
    Closed
    aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
    @websurfer5 @serhiy-storchaka @aisk
    pythongh-81441: shutil.rmtree() FileNotFoundError race condition (pyt…
    1f0336c 
    …honGH-14064)

Ignore missing files and directories while enumerating
directory entries in shutil.rmtree().

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @iritkatriel @serhiy-storchaka