This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: shutil.copytree behavior is inconsistent with copyfile
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 3.3, Python 2.7, Python 2.6
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: brian.curtin, joseph.h.garvin, pitrou
Priority: normal Keywords:

Created on 2010-03-12 16:25 by joseph.h.garvin, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (4)
msg100943 - (view) Author: (joseph.h.garvin) Date: 2010-03-12 16:25
shutil.copyfile's behavior is to replace the dst file if it already exists. shutil.copytree requires that the destination not already exist, and throws an OSError if it does. I see 3 problems with this behavior:

1. It's inconsistent with copyfile

2. It's inconsistent with 'cp', which is what users would be using if they were writing shell script. Given shutil's namesake I assume it's supposed to help make python a viable shell script replacement.

3. It makes it difficult to use copytree with tempfile.mkdtemp(). If I want to make a temporary directory and copy a folder into it, I need to copytree to a nonexist subfolder then move all the files down a level or resort to other unpythonic hacks. In my project I copy unit test resources to a temp folder because they're manipulated throughout the test and I want to keep the originals. I imagine that this is a common use case.

That said, if anyone depended on shutil.copytree failing to test whether or not a folder exists, changing this would break compatibility, so if it can't be changed, maybe a shutil.mergetree would be appropriate.
msg100944 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-03-12 16:36
> 1. It's inconsistent with copyfile

I think it's ok. When using copyfile you can easily check that the destination file doesn't exist, while with copytree it's harder to do the same for *all* destination files. Therefore it makes sense for copytree to raise an error in this case.

> Given shutil's namesake I assume it's supposed to help make python a viable shell script replacement.

It doesn't mean the semantics have to be exactly the same. Actually, many Python users are under Windows where semantics will be different.

> It makes it difficult to use copytree with tempfile.mkdtemp(). If I want to make a temporary directory and copy a folder into it, I need to copytree to a nonexist subfolder then move all the files down a level or resort to other unpythonic hacks.

I actually don't understand your problem. If you are creating a temporary directory, it is empty so there isn't any risk of hitting an already existing file.

In any case, I don't think your arguments are good enough to warrant breaking compatibility with older versions.
msg100945 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-03-12 16:37
Also, keep in mind that shutil.copytree is more of an example than the be-all-end-all directory copy function, as stated in both the documentation and source code.
msg100981 - (view) Author: (joseph.h.garvin) Date: 2010-03-12 23:42
Sorry I wasn't terribly clear explaining the problem with mkdtemp. Say you create your temporary folder, /tmp/foo. Now you have a regular folder bar laid out like so:

bar/
  blarg1.ext
  blarg2.ext
  subdir/
    blarg3.ext
    blarg4.ext

I'd like this to be the result of copying:

/tmp
  /foo
    blarg1.ext
    blarg2.ext
    subdir/
      blarg3.ext
      blarg4.ext

Basically, I'd like to copy bar/ to /tmp/foo such that the paths of the files relative to bar/ are the same as the paths of the copied files relative to /tmp/foo. AFAICT there's no easy way to do this with copytree's current behavior. Because tempfile.mkdtemp() creates /tmp/foo to start, you can't use copytree to do this. 

But now I realize copy tree wants to copy the source directory as a directory, so you always get /tmp/foo/bar. What I wanted was copy(recursive_glob(src, "*"), dst). That would give you the effect you normally get from a utility like 'cp'.

> It doesn't mean the semantics have to be exactly the same. Actually, many Python users are under Windows where semantics will be different.

I believe 'copy' has the same cp-like semantics I desire under Windows, but it's been quite some time since I used it.

So, not a bug, but sticks out to me as missing. Might make sense someday to have a keyword arg to copytree that would make it behave this way.
History
Date User Action Args
2022-04-11 14:56:58adminsetgithub: 52372
2010-03-12 23:42:35joseph.h.garvinsetmessages: + msg100981
2010-03-12 16:37:57brian.curtinsetnosy: + brian.curtin
messages: + msg100945
2010-03-12 16:36:03pitrousetstatus: open -> closed

nosy: + pitrou
messages: + msg100944

resolution: rejected
2010-03-12 16:28:12joseph.h.garvinsettype: behavior
2010-03-12 16:25:06joseph.h.garvincreate