classification
Title: shutil.copystat utime lookup fails on certain Android file systems
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Jerry A, iritkatriel
Priority: normal Keywords:

Created on 2016-09-14 06:04 by Jerry A, last changed 2021-12-29 22:34 by iritkatriel. This issue is now closed.

Files
File name Uploaded Description Edit
copyoops2.py Jerry A, 2016-09-14 06:04 reproduce the utime failure of copystat
Messages (3)
msg276385 - (view) Author: Jerry A (Jerry A) Date: 2016-09-14 06:04
shutil.copystat fails on certain Android filesystems when trying lookup("utime") (line 194 of https://hg.python.org/cpython/file/3.5/Lib/shutil.py)

Using Python 2.7 and Python 3.5 on Android 6.0.1., when I am on the "typical" user file system, and I run a script that uses shutil.copy2, that copy2 will fail when it calls shutil.copystat and that copystat fails performing utime on the destination file.

I am attaching a script that can be used to reproduce this. I run the script using python 3.5 within termux.

Below is a transcript my running of the attacked script.

$ pwd
/data/data/com.termux/files/home

$ ls --l
Dropbox  storage  tmp  venv  workspace

$ ls -la
total 52
drwx------ 10 u0_a73 u0_a73    4096 Sep 13 16:07 .
drwx------  4 u0_a73 u0_a73    4096 Jul 10 02:18 ..
-rw-------  1 u0_a73 u0_a73    7011 Sep 13 20:34 .bash_history
drwx------  3 u0_a73 u0_a73    4096 Jul 10 02:30 .cache
drwxrwx--- 12 u0_a73 everybody 4096 Sep 13 15:35 .emacs.d
drwx------  2 u0_a73 u0_a73    4096 Jul 10 15:58 .gnupg
-rw-------  1 u0_a73 u0_a73     145 Sep 13 16:07 .python_history
drwx------  2 u0_a73 everybody 4096 Sep 13 12:45 .ssh
drwx------  3 u0_a73 u0_a73    4096 Sep 13 14:10 Dropbox
drwx------  2 u0_a73 u0_a73    4096 Jul 10 03:42 storage
drwx------  2 u0_a73 u0_a73    4096 Sep 13 14:24 tmp
drwx------  5 u0_a73 u0_a73    4096 Jul 10 02:33 venv
lrwxrwxrwx  1 u0_a73 u0_a73      30 Jul 10 15:54 workspace -> /storage/emulated/0/workspace/


$ cat /proc/mounts
rootfs / rootfs ro,seclabel,relatime 0 0
tmpfs /dev tmpfs rw,seclabel,nosuid,relatime,mode=755 0 0
devpts /dev/pts devpts rw,seclabel,relatime,mode=600 0 0
none /dev/memcg cgroup rw,relatime,memory 0 0
none /dev/cpuctl cgroup rw,relatime,cpu 0 0
adb /dev/usb-ffs/adb functionfs rw,relatime 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,seclabel,relatime 0 0
selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0
none /sys/fs/cgroup tmpfs rw,seclabel,relatime,mode=750,gid=1000 0 0
none /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0
/sys/kernel/debug /sys/kernel/debug debugfs rw,seclabel,relatime 0 0
none /acct cgroup rw,relatime,cpuacct 0 0
tmpfs /mnt tmpfs rw,seclabel,relatime,mode=755,gid=1000 0 0
/dev/fuse /mnt/runtime/default/emulated fuse rw,nosuid,nodev,noexec,noatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0
/dev/fuse /mnt/runtime/read/emulated fuse rw,nosuid,nodev,noexec,noatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0
/dev/fuse /mnt/runtime/write/emulated fuse rw,nosuid,nodev,noexec,noatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0
/dev/block/platform/dw_mmc.0/by-name/system /system ext4 ro,seclabel,relatime 0 0
/dev/block/platform/dw_mmc.0/by-name/cache /cache ext4 rw,seclabel,nosuid,nodev,noatime,journal_checksum,journal_async_commit,noauto_da_alloc,errors=panic,data=ordered 0 0
/dev/block/platform/dw_mmc.0/by-name/userdata /data ext4 rw,seclabel,nosuid,nodev,noatime,journal_checksum,journal_async_commit,noauto_da_alloc,errors=panic,data=ordered 0 0
/dev/block/platform/dw_mmc.0/by-name/efs /factory ext4 ro,seclabel,relatime,data=ordered 0 0
tmpfs /storage tmpfs rw,seclabel,relatime,mode=755,gid=1000 0 0
/dev/fuse /storage/emulated fuse rw,nosuid,nodev,noexec,noatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0
tmpfs /storage/self tmpfs rw,seclabel,relatime,mode=755,gid=1000 0 0
$

$ cd workspace
$ cd foo
$ cd copystatbug/

$ echo "this is Mr. Lorem Ipsum" > foo.bar
$ cat foo.bar
this is Mr. Lorem Ipsum

$ python copyoops2.py
copyfoo
A problem lookingup utime(<built-in function utime>) of foo.foo; [Errno 1] Operation not permitted
A problem copying the file metadata of foo.bar to foo.foo; [Errno 1] Operation not permitted: 'foo.foo'
copybar
A problem lookingup utime(<built-in function utime>) of bar.bar; [Errno 1] Operation not permitted
Traceback (most recent call last):
  File "copyoops2.py", line 155, in <module>
    copybar("foo.bar", "bar.bar")
  File "copyoops2.py", line 151, in copybar
    copystat(src, dst, follow_symlinks=follow_symlinks)
  File "copyoops2.py", line 81, in copystat
    lookup("chmod")(dst, mode, follow_symlinks=follow)
PermissionError: [Errno 1] Operation not permitted: 'bar.bar'
$ ls -la
total 36
drwxrwx--- 2 root everybody 4096 Sep 13 22:40 .
drwxrwx--- 4 root everybody 4096 Sep 13 20:42 ..
-rw-rw---- 1 root everybody   24 Sep 13 22:40 bar.bar
-rw-rw---- 1 root everybody 4846 Sep 13 22:38 copyoops.py
-rw-rw---- 1 root everybody 4838 Sep 13 22:39 copyoops2.py
-rw-rw---- 1 root everybody   24 Sep 13 20:57 foo.bar
-rw-rw---- 1 root everybody   24 Sep 13 22:40 foo.foo
$ python --version
Python 3.5.2
msg407931 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-07 12:58
Jerry, is this still a problem with current python versions? (2.7/3.5 are no longer being maintained).
msg409330 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-29 22:34
Please create a new issue if this is still a problem on a currentl version of python (>= 3.9).
History
Date User Action Args
2021-12-29 22:34:48iritkatrielsetstatus: pending -> closed
resolution: out of date
messages: + msg409330

stage: resolved
2021-12-07 12:58:40iritkatrielsetstatus: open -> pending
nosy: + iritkatriel
messages: + msg407931

2016-09-14 06:04:10Jerry Acreate