classification
Title: Add a "force" parameter to shutil.rmtree
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: duplicate
Dependencies: Superseder: shutil rmtree fails on readonly files in Windows
View: 19643
Assigned To: Nosy List: Nirbheek Chauhan, neologix, paul.moore, r.david.murray, zach.ware
Priority: normal Keywords:

Created on 2014-07-22 19:54 by paul.moore, last changed 2016-05-19 05:01 by Nirbheek Chauhan. This issue is now closed.

Messages (8)
msg223685 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2014-07-22 19:54
It would be useful for shutil.rmtree to have a "force" argument that overrode read-only permission issues, essentially replicating the behaviour of the -f flag in rm -rf (Unix) and the -force parameter of del (Windows Powershell).

It's possible to use the onerror callback to implement this, but it's tricky to get right in a cross-platform manner. See http://stackoverflow.com/questions/2656322, which recommends

def onerror(func, path, exc_info):
    if not os.access(path, os.W_OK):
        os.chmod(path, stat.S_IWUSR)
        func(path)
    else:
        raise

and http://stackoverflow.com/questions/1889597 which recommends

def remove_readonly(func, path, excinfo):
    os.chmod(path, stat.S_IWRITE)
    func(path)

It's not clear whether either of these is portable, though (the former looks to me like it's Unix-specific and the latter like it's for Windows, but I'm not sure).

Having the functionality available in the standard library function directly avoids having people write tricky and potentially buggy code for what is a pretty common situation. (In particular, this comes up a lot in code that deletes git checkouts on Windows, where git makes parts of the .git directory readonly).
msg223686 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2014-07-22 20:02
On Unix at least, this doesn't apply: rm -f doesn't mean "ignore permissions", but but rather don't ask confirmation which the rm commands asks in some cases (empty file, directory, etc).
Ans the code posted wouldn't work, since the permission to remove a file applies to to the file, but to the parent directory (must have write permission). And we certainly don't want to change the directory permission anyway.

Not sure about Windows, though.
msg223687 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-07-22 20:12
Actually it does apply on unix:

rdmurray@session:~/tmp>ls -ld foo
drwxr-x--- 2 rdmurray rdmurray 4096 Jul 22 16:09 foo
rdmurray@session:~/tmp>ls -l foo
total 0
-r--r----- 1 rdmurray rdmurray 0 Jul 22 16:09 bar
rdmurray@session:~/tmp>rm -r foo
rm: remove write-protected regular empty file ‘foo/bar’? 

If I say yes it will remove it, since I have write perms on the directory.  rm -rf will not prompt and will delete it.

I believe the situation is analogous on Windows, but I"m not sure.
msg223689 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2014-07-22 20:20
> Actually it does apply on unix:

No, it does not apply: here's what I've written:
"""
 rm -f doesn't mean "ignore permissions", but but rather don't ask
confirmation which the rm commands asks in some cases (empty file,
directory, etc).
"""

Having a file non-writable fits in those "some cases" where rm is
extra careful and asks confirmation.: calling shutil.rmtree() on your
example will succeed, so it does not apply on Unix.
msg223694 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-07-22 20:36
In other words, on unix shutil.rmtree is *already* 'rm -rf'.  This then argues that it *not* deleting read only files on Windows is a bug, albeit one we may not be able to fix for backward compatibility reasons.

Ah, and now my memory is jogged.  This is a duplicate of issue 19643, and that was more or less the conclusion.
msg223702 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2014-07-22 21:42
Doh. And I was even involved in the previous issue. Sorry for the noise.
msg223703 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2014-07-22 21:49
> In other words, on unix shutil.rmtree is *already* 'rm -rf'.

Exactly :-)
msg265846 - (view) Author: Nirbheek Chauhan (Nirbheek Chauhan) Date: 2016-05-19 05:01
> In other words, on unix shutil.rmtree is *already* 'rm -rf'.

This is not true. See:

  $ mkdir testdir && chmod 200 testdir && ls -lhd testdir
  d-w------- 2 nirbheek nirbheek 4.0K May 19 10:21 testdir

`rm -rf` works fine on this. But shutil.rmtree borks:

$ python3 -c 'import shutil; shutil.rmtree("testdir")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.5/shutil.py", line 470, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/usr/lib64/python3.5/shutil.py", line 468, in rmtree
    fd = os.open(path, os.O_RDONLY)
PermissionError: [Errno 13] Permission denied: 'testdir'

The -f option to `rm` ensures that it tries its hardest to resolve permissions problems and does not error out if it can't resolve them either. The latter is available via 'ignore_errors', but the former is a missing feature. A shutil.rmtree flag that 'resolves permissions' would be useful on all platforms. Not just Windows.
History
Date User Action Args
2016-05-19 05:01:24Nirbheek Chauhansetnosy: + Nirbheek Chauhan
messages: + msg265846
2014-07-22 21:49:54neologixsetmessages: + msg223703
2014-07-22 21:42:32paul.mooresetmessages: + msg223702
2014-07-22 20:36:43r.david.murraysetstatus: open -> closed
superseder: shutil rmtree fails on readonly files in Windows
messages: + msg223694

resolution: duplicate
stage: resolved
2014-07-22 20:20:18neologixsetmessages: + msg223689
2014-07-22 20:12:24r.david.murraysetnosy: + r.david.murray, zach.ware
messages: + msg223687
2014-07-22 20:02:33neologixsetnosy: + neologix
messages: + msg223686
2014-07-22 19:54:30paul.moorecreate