Title: shutil.move raises OSError when copystat fails
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jeffrey.Kintscher, eric.araujo, eshan-singhal, piman, thomaswaldmann
Priority: normal Keywords: patch

Created on 2006-02-25 05:26 by piman, last changed 2021-12-26 15:58 by ajaksu2.

File name Uploaded Description Edit piman, 2006-02-26 17:56 Fix copy2/copy
Messages (6)
msg27612 - (view) Author: Joe Wreschnig (piman) Date: 2006-02-25 05:26
If you are on a Linux system, and shutil.move a file
from anywhere onto a partition where you have write
permission but are not the owner, os.utime will fail
with an EPERM OSError. This can (and did) happen moving
a file on a vfat partition mounted with umask=0000, so
every user had read/write/execute but all files were
owned by root.

This happens in shutil.copystat, so shutil.move doesn't
remove the old file. The resulting error code (OSError,
EPERM) is not distinguishable from several other
permission errors that can happen during shutil.move,
even though a failure to set a utime is not fatal for
most move operations (mv(1) succeeds, for example).

I would suggest either ignoring an EPERM from copystat,
or catching it in shutil.copy2 and raising a more
specific exception so that it can be easily
distinguished from genuine failure due to permissions.
msg27613 - (view) Author: Joe Wreschnig (piman) Date: 2006-02-26 05:08
Logged In: YES 

The attached patch causes shutil.copy and shutil.copy2 (and
so copytree, and move) to ignore EACCES and EPERM from
copymode and copystat respectively. Other errors are
reraised with the original traceback. Any error copying the
file, rather than the metadata, is still considered fatal.
User calls to copymode and copystat still fail as before.

Justification: On Linux, utime(2) says the distinction
between EACCES and EPERM is blurry and "Linux is not careful
to distinguish between the EACCES and EPERM" so the patch
ignores both. In my opinion, anyone wanting to deal with
low-level details like mtimes and modes is probably not
using shutil, so ignoring is better than a new exception
(which would just result in complicated new code, and
surprise old code). It's also easy for a caller to check if
the mode/utimes were copied, if it's important to them.
msg27614 - (view) Author: Thomas Waldmann (thomaswaldmann) Date: 2007-03-24 18:51
See also for a similar problem with copystat.

msg141827 - (view) Author: √Čric Araujo (eric.araujo) * (Python committer) Date: 2011-08-09 15:36
> In my opinion, anyone wanting to deal with low-level details like
> mtimes and modes is probably not using shutil

As a moderately experienced UNIX user, I would say that file permissions are not low-level details.  I see shutil functions as convenient wrappers to automate looping and calling a function, but I do care about file permissions.

If people are still interested in this modification, I think it would need a backward-compatible change, for example in the form of a new argument ignore_permission_errors, or a more generic onerror callback like rmtree has.
msg221789 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-28 14:05
The patch would need changing to allow for the follow_symlinks parameter and the backward compatibility issues mention in msg141827.  Do we wait for an updated patch, close as "won't fix" or what?
msg358270 - (view) Author: Eshan Singhal (eshan-singhal) * Date: 2019-12-11 16:18
Amending shutil.copy and shutil.copy2 to expose a method of ignoring the permission errors rather than changing the current behaviour would not fix the original issue without further changes in consumers of these functions to pass through the new parameter (i.e. shutil.move in this scenario).

This may be undesirable as copy_function is an optional parameter to shutil.move and so there would have to be a specialcase guard/check to see if the copy_function is shutil.copy or shutil.copy2 when passing in a new ignore_permission_errors parameter.

I propose a new issue to track adding functionality to optionally ignore permission issues for shutil.copy/shutil.copy2, and this issue can remain open.
Date User Action Args
2021-12-26 15:58:26ajaksu2setkeywords: - easy
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.3
2019-12-11 16:18:10eshan-singhalsetnosy: + eshan-singhal
messages: + msg358270
2019-06-02 07:38:22Jeffrey.Kintschersetnosy: + Jeffrey.Kintscher
2019-03-16 00:01:19BreamoreBoysetnosy: - BreamoreBoy
2014-06-28 14:05:35BreamoreBoysetnosy: + BreamoreBoy
messages: + msg221789
2011-08-09 15:36:21eric.araujosetnosy: + eric.araujo

messages: + msg141827
versions: + Python 3.3, - Python 3.1, Python 2.7, Python 3.2
2010-08-26 16:38:07BreamoreBoysetstage: test needed -> patch review
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 3.0
2009-04-22 14:35:32ajaksu2setkeywords: + easy
2009-03-21 00:36:36ajaksu2setkeywords: + patch
stage: test needed
type: behavior
versions: + Python 2.6, Python 3.0, - Python 2.4
2006-02-25 05:26:29pimancreate