Message231745
To see what happens at syscall level, I've run various implementations
of get_tree_size() functions (see get_tree_size_listdir.diff) with
strace:
get_tree_size_listdir_fd -- os.listdir(fd) + os.lstat
get_tree_size -- os.scandir(path) + entry.lstat
get_tree_size_listdir_stat -- os.listdir(path) + os.lstat
get_tree_size_listdir -- os.listdir(path) + os.path.isdir + os.lstat
Summary:
- os.listdir() and os.scandir()-based variants use the same number of getdents() syscalls
- and the number of openat, lstat (newfstatat) syscalls is also comparable
(except for the variant that uses os.path.isdir)
Log:
scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_fd as f; import os; print(f(os.open("/usr/", os.O_RDONLY)))' && head -7 py.strace
5535240217
11.29user 8.14system 0:17.78elapsed 109%CPU (0avgtext+0avgdata 13460maxresident)k
0inputs+8outputs (0major+6781minor)pagefaults 0swaps
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
46.51 0.075252 0 264839 newfstatat
16.88 0.027315 1 50460 getdents
11.53 0.018660 0 75621 fcntl
9.74 0.015758 0 50531 close
6.87 0.011116 0 25214 openat
scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size as f; print(f("/usr/"))' && head -7 py.strace
5535240217
22.56user 8.47system 0:29.77elapsed 104%CPU (0avgtext+0avgdata 13280maxresident)k
0inputs+8outputs (0major+6306minor)pagefaults 0swaps
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
62.00 0.032822 0 239644 lstat
19.97 0.010570 0 50460 getdents
8.52 0.004510 0 25215 openat
6.09 0.003224 0 25326 close
0.55 0.000292 3 95 mmap
scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_stat as f; print(f("/usr/"))' && head -7 py.strace
5535240217
13.70user 6.30system 0:18.84elapsed 106%CPU (0avgtext+0avgdata 13456maxresident)k
0inputs+8outputs (0major+6769minor)pagefaults 0swaps
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
64.79 0.050217 0 264849 lstat
19.37 0.015011 0 50460 getdents
8.22 0.006367 0 25215 openat
5.76 0.004465 0 25326 close
0.32 0.000247 2 114 mmap
scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir as f; print(f("/usr/"))' && head -7 py.strace
5535240217
19.53user 10.61system 0:28.16elapsed 107%CPU (0avgtext+0avgdata 13452maxresident)k
0inputs+8outputs (0major+6733minor)pagefaults 0swaps
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
42.04 0.063050 0 265195 37259 stat
37.40 0.056086 0 265381 lstat
11.46 0.017187 0 50460 getdents
4.82 0.007232 0 25215 openat
3.43 0.005139 0 25326 close |
|
Date |
User |
Action |
Args |
2014-11-27 04:31:13 | akira | set | recipients:
+ akira, tebeka, pitrou, vstinner, giampaolo.rodola, tim.golden, benhoyt, abacabadabacaba, socketpair, josh.r |
2014-11-27 04:31:13 | akira | set | messageid: <1417062673.14.0.908320154024.issue22524@psf.upfronthosting.co.za> |
2014-11-27 04:31:13 | akira | link | issue22524 messages |
2014-11-27 04:31:12 | akira | create | |
|