This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Add copy_directory_metadata parameter to shutil.copytree
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: desbma, giampaolo.rodola
Priority: normal Keywords: patch

Created on 2017-11-18 19:13 by desbma, last changed 2022-04-11 14:58 by admin.

Pull Requests
URL Status Linked Edit
PR 4456 open desbma, 2017-11-18 19:27
Messages (7)
msg306494 - (view) Author: desbma (desbma) * Date: 2017-11-18 19:13
I am sometimes using shutil.copytree to copy a directory to a destination that does not support setting metadata (like MTP mounts of Android devices).

Using the copy_function parameter allows passing shutil.copy or a custom function to ignore file metadata, however currently shutil.copytree always tries to call copystat on directories, which can fail with OSError (errno set to ENOTSUPP).

The code assumes copystat can fail on Windows, but propagates the error on other OS, even though the tree copy actually succeeds.
msg308628 - (view) Author: desbma (desbma) * Date: 2017-12-19 11:49
msg323055 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2018-08-03 14:36
What function raises ENOTSUPP exactly (traceback would be welcome)?
msg323075 - (view) Author: desbma (desbma) * Date: 2018-08-03 20:56
Traceback is not very useful in that case:

mkdir /tmp/a
touch /tmp/a/b 

Python 3.6.6 (default, Jun 27 2018, 13:11:40) 
[GCC 8.1.1 20180531] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, shutil
>>> os.getcwd()
>>> shutil.copytree("/tmp/a", os.path.join(os.getcwd(), "test"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/desbma/py-venvs/main/lib/python3.6/", line 359, in copytree
    raise Error(errors)
shutil.Error: [('/tmp/a/b', '/run/user/1000/gvfs/mtp:host=%5Busb%3A001%2C006%5D/sdcard1/test/b', '[Errno 95] Operation not supported'), ('/tmp/a', '/run/user/1000/gvfs/mtp:host=%5Busb%3A001%2C006%5D/sdcard1/test', '[Errno 95] Operation not supported')]
>>> shutil.copytree("/tmp/a", os.path.join(os.getcwd(), "test2"), copy_function=shutil.copy)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/desbma/py-venvs/main/lib/python3.6/", line 359, in copytree
    raise Error(errors)
shutil.Error: [('/tmp/a/b', '/run/user/1000/gvfs/mtp:host=%5Busb%3A001%2C006%5D/sdcard1/test2/b', "[Errno 95] Operation not supported: '/run/user/1000/gvfs/mtp:host=%5Busb%3A001%2C006%5D/sdcard1/test2/b'"), ('/tmp/a', '/run/user/1000/gvfs/mtp:host=%5Busb%3A001%2C006%5D/sdcard1/test2', '[Errno 95] Operation not supported')]

The exception is thrown from here
msg323077 - (view) Author: desbma (desbma) * Date: 2018-08-03 21:01
Note that in the examples above both copytree calls actually succeed (only metadata copy failed).

The user can disable file metadata copy by passing 'copy_function=shutil.copy', but there is no way to do the same for directories and the directory copystat call fails.
msg323081 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2018-08-03 21:25
It's not clear where the exception originates from. Try to use copy2() instead of copytree().
msg323082 - (view) Author: desbma (desbma) * Date: 2018-08-03 21:34
copy2 always raises "OSError: [Errno 95] Operation not supported" here but I can work around that by passing copy_function=shutil.copy to copytree as I did above.

The problem is that there is currently no way to avoid the copystat call made on the directory itself.
Date User Action Args
2022-04-11 14:58:54adminsetgithub: 76254
2018-08-03 21:34:55desbmasetmessages: + msg323082
2018-08-03 21:25:43giampaolo.rodolasetmessages: + msg323081
2018-08-03 21:01:03desbmasetmessages: + msg323077
2018-08-03 20:56:38desbmasetmessages: + msg323075
2018-08-03 14:36:57giampaolo.rodolasetmessages: + msg323055
2018-06-12 09:41:30giampaolo.rodolasetnosy: + giampaolo.rodola
2017-12-19 11:49:25desbmasetmessages: + msg308628
2017-11-18 19:27:13desbmasetkeywords: + patch
stage: patch review
pull_requests: + pull_request4394
2017-11-18 19:13:27desbmacreate