Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a "force" parameter to shutil.rmtree #66239

Closed
pfmoore opened this issue Jul 22, 2014 · 8 comments
Closed

Add a "force" parameter to shutil.rmtree #66239

pfmoore opened this issue Jul 22, 2014 · 8 comments
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@pfmoore
Copy link
Member

pfmoore commented Jul 22, 2014

BPO 22040
Nosy @pfmoore, @bitdancer, @zware
Superseder
  • bpo-19643: shutil rmtree fails on readonly files in Windows
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2014-07-22.20:36:43.938>
    created_at = <Date 2014-07-22.19:54:30.645>
    labels = ['type-feature', 'library']
    title = 'Add a "force" parameter to shutil.rmtree'
    updated_at = <Date 2016-05-19.05:01:24.746>
    user = 'https://github.com/pfmoore'

    bugs.python.org fields:

    activity = <Date 2016-05-19.05:01:24.746>
    actor = 'Nirbheek Chauhan'
    assignee = 'none'
    closed = True
    closed_date = <Date 2014-07-22.20:36:43.938>
    closer = 'r.david.murray'
    components = ['Library (Lib)']
    creation = <Date 2014-07-22.19:54:30.645>
    creator = 'paul.moore'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 22040
    keywords = []
    message_count = 8.0
    messages = ['223685', '223686', '223687', '223689', '223694', '223702', '223703', '265846']
    nosy_count = 5.0
    nosy_names = ['paul.moore', 'r.david.murray', 'neologix', 'zach.ware', 'Nirbheek Chauhan']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '19643'
    type = 'enhancement'
    url = 'https://bugs.python.org/issue22040'
    versions = ['Python 3.5']

    @pfmoore
    Copy link
    Member Author

    pfmoore commented Jul 22, 2014

    It would be useful for shutil.rmtree to have a "force" argument that overrode read-only permission issues, essentially replicating the behaviour of the -f flag in rm -rf (Unix) and the -force parameter of del (Windows Powershell).

    It's possible to use the onerror callback to implement this, but it's tricky to get right in a cross-platform manner. See http://stackoverflow.com/questions/2656322, which recommends

    def onerror(func, path, exc_info):
        if not os.access(path, os.W_OK):
            os.chmod(path, stat.S_IWUSR)
            func(path)
        else:
            raise

    and http://stackoverflow.com/questions/1889597 which recommends

    def remove_readonly(func, path, excinfo):
        os.chmod(path, stat.S_IWRITE)
        func(path)

    It's not clear whether either of these is portable, though (the former looks to me like it's Unix-specific and the latter like it's for Windows, but I'm not sure).

    Having the functionality available in the standard library function directly avoids having people write tricky and potentially buggy code for what is a pretty common situation. (In particular, this comes up a lot in code that deletes git checkouts on Windows, where git makes parts of the .git directory readonly).

    @pfmoore pfmoore added stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Jul 22, 2014
    @neologix
    Copy link
    Mannequin

    neologix mannequin commented Jul 22, 2014

    On Unix at least, this doesn't apply: rm -f doesn't mean "ignore permissions", but but rather don't ask confirmation which the rm commands asks in some cases (empty file, directory, etc).
    Ans the code posted wouldn't work, since the permission to remove a file applies to to the file, but to the parent directory (must have write permission). And we certainly don't want to change the directory permission anyway.

    Not sure about Windows, though.

    @bitdancer
    Copy link
    Member

    Actually it does apply on unix:

    rdmurray@session:/tmp>ls -ld foo
    drwxr-x--- 2 rdmurray rdmurray 4096 Jul 22 16:09 foo
    rdmurray@session:
    /tmp>ls -l foo
    total 0
    -r--r----- 1 rdmurray rdmurray 0 Jul 22 16:09 bar
    rdmurray@session:~/tmp>rm -r foo
    rm: remove write-protected regular empty file ‘foo/bar’?

    If I say yes it will remove it, since I have write perms on the directory. rm -rf will not prompt and will delete it.

    I believe the situation is analogous on Windows, but I"m not sure.

    @neologix
    Copy link
    Mannequin

    neologix mannequin commented Jul 22, 2014

    Actually it does apply on unix:

    No, it does not apply: here's what I've written:
    """
    rm -f doesn't mean "ignore permissions", but but rather don't ask
    confirmation which the rm commands asks in some cases (empty file,
    directory, etc).
    """

    Having a file non-writable fits in those "some cases" where rm is
    extra careful and asks confirmation.: calling shutil.rmtree() on your
    example will succeed, so it does not apply on Unix.

    @bitdancer
    Copy link
    Member

    In other words, on unix shutil.rmtree is *already* 'rm -rf'. This then argues that it *not* deleting read only files on Windows is a bug, albeit one we may not be able to fix for backward compatibility reasons.

    Ah, and now my memory is jogged. This is a duplicate of bpo-19643, and that was more or less the conclusion.

    @pfmoore
    Copy link
    Member Author

    pfmoore commented Jul 22, 2014

    Doh. And I was even involved in the previous issue. Sorry for the noise.

    @neologix
    Copy link
    Mannequin

    neologix mannequin commented Jul 22, 2014

    In other words, on unix shutil.rmtree is *already* 'rm -rf'.

    Exactly :-)

    @NirbheekChauhan
    Copy link
    Mannequin

    NirbheekChauhan mannequin commented May 19, 2016

    In other words, on unix shutil.rmtree is *already* 'rm -rf'.

    This is not true. See:

      $ mkdir testdir && chmod 200 testdir && ls -lhd testdir
      d-w------- 2 nirbheek nirbheek 4.0K May 19 10:21 testdir

    rm -rf works fine on this. But shutil.rmtree borks:

    $ python3 -c 'import shutil; shutil.rmtree("testdir")'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/lib64/python3.5/shutil.py", line 470, in rmtree
        onerror(os.lstat, path, sys.exc_info())
      File "/usr/lib64/python3.5/shutil.py", line 468, in rmtree
        fd = os.open(path, os.O_RDONLY)
    PermissionError: [Errno 13] Permission denied: 'testdir'

    The -f option to rm ensures that it tries its hardest to resolve permissions problems and does not error out if it can't resolve them either. The latter is available via 'ignore_errors', but the former is a missing feature. A shutil.rmtree flag that 'resolves permissions' would be useful on all platforms. Not just Windows.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants