classification
Title: shutil.copytree failing on xattr-less filesystems (like NFS)
Type: behavior Stage: patch review
Components: Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: hynek Nosy List: hynek, python-dev, twouters
Priority: normal Keywords: needs review, patch

Created on 2013-01-29 18:18 by twouters, last changed 2013-02-05 08:16 by hynek. This issue is now closed.

Files
File name Uploaded Description Edit
shutil.patch twouters, 2013-01-31 21:44 shutil._coypxattr patch with test review
Messages (6)
msg180920 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2013-01-29 18:18
The new xattr support in shutil causes shutil.copytree and shutil.copy2 to fail inelegantly on (source) filesystems that do not support xattrs (like NFS):

# /home/twouters does not support xattrs
>>> os.listxattr("/home/twouters/foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 95] Operation not supported: '/home/twouters/foo'

>>> shutil.copytree("/home/twouters/spam", "/tmp/spam")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/twouters/gvs-pristine/python/Python-3.3.0/Lib/shutil.py", line 345, in copytree
    raise Error(errors)
shutil.Error: [('/home/twouters/spam/ham', '/tmp/spam/ham', "[Errno 95] Operation not supported: '/home/twouters/spam/ham'"), ('/home/twouters/spam/eggs', '/tmp/spam/eggs', "[Errno 95] Operation not supported: '/home/twouters/spam/eggs'"), ('/home/twouters/spam', '/tmp/spam', "[Errno 95] Operation not supported: '/home/twouters/spam'")]

(The actual files will have been copied, since xattr copies are done after everything else.)  Interestingly shutil._copyxattr does try to cope with unsupported xattrs on the *target* filesystem (which seems like it might be a mistake to do, since it loses data), just not the original filesystem (which seems like a sensible thing instead):

# /tmp does support xattrs
>>> os.listxattr("/tmp/spam")
[]
>>> shutil.copytree("/tmp/spam", "/home/twouters/spam-new")
'/home/twouters/spam-new'

The attached patch fixes shutil._copyxattr to also ignore unsupported/empty xattrs (but not permission errors) on the source. (I'm not certain if errno.ENODATA can be expected from os.listxattr(), but internet searches suggest that some people think so, and I don't know what other meaning it could have.)
msg180947 - (view) Author: Hynek Schlawack (hynek) * (Python committer) Date: 2013-01-29 21:01
Could you add regression tests to your patch please?
msg181046 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2013-01-31 21:42
Updated patch with test.
msg181047 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2013-01-31 21:44
Now really updated the patch.
msg181423 - (view) Author: Roundup Robot (python-dev) Date: 2013-02-05 07:26
New changeset 47c65639390d by Hynek Schlawack in branch '3.3':
#17076: Make copying of xattrs more permissive of missing FS support
http://hg.python.org/cpython/rev/47c65639390d

New changeset 7ccdbd1cd213 by Hynek Schlawack in branch 'default':
#17076: Make copying of xattrs more permissive of missing FS support
http://hg.python.org/cpython/rev/7ccdbd1cd213
msg181424 - (view) Author: Hynek Schlawack (hynek) * (Python committer) Date: 2013-02-05 08:16
The buildbots look happy, thank you for spotting & the patch Thomas!
History
Date User Action Args
2013-02-05 08:16:53hyneksetstatus: open -> closed
resolution: fixed
messages: + msg181424
2013-02-05 07:26:20python-devsetnosy: + python-dev
messages: + msg181423
2013-02-05 07:24:50hyneksetversions: + Python 3.4
2013-01-31 21:44:16twouterssetfiles: + shutil.patch

messages: + msg181047
2013-01-31 21:43:37twouterssetfiles: - shutil.patch
2013-01-31 21:43:16twouterssetfiles: - shutil.patch
2013-01-31 21:42:58twouterssetfiles: + shutil.patch

messages: + msg181046
2013-01-29 21:01:20hyneksetpriority: high -> normal
assignee: hynek
type: crash -> behavior
messages: + msg180947
2013-01-29 18:57:02pitrousetnosy: + hynek
2013-01-29 18:18:12twouterscreate