Message333819
In case close_fds=True is passed to subprocess.Popen()
or its users (subprocess.call() etc), it might spend
some considerable time closing non-opened file descriptors,
as demonstrated by the following snippet from strace:
close(3) = -1 EBADF (Bad file descriptor)
close(5) = -1 EBADF (Bad file descriptor)
close(6) = -1 EBADF (Bad file descriptor)
close(7) = -1 EBADF (Bad file descriptor)
...
close(1021) = -1 EBADF (Bad file descriptor)
close(1022) = -1 EBADF (Bad file descriptor)
close(1023) = -1 EBADF (Bad file descriptor)
This happens because the code in _close_fds() iterates from 3 up to
MAX_FDS = os.sysconf("SC_OPEN_MAX").
Now, syscalls are cheap, but SC_OPEN_MAX (also known as RLIMIT_NOFILE
or ulimit -n) can be quite high, for example:
$ docker run --rm python python3 -c \
$'import os\nprint(os.sysconf("SC_OPEN_MAX"))'
1048576
This means a million syscalls before spawning a child process, which
can result in a major delay, like 0.1s as measured on my fast and mostly
idling laptop. Here is the comparison with python3 (which does not have this problem):
$ docker run --rm python python3 -c $'import subprocess\nimport time\ns = time.time()\nsubprocess.check_call([\'/bin/true\'], close_fds=True)\nprint(time.time() - s)\n'
0.0009245872497558594
$ docker run --rm python python2 -c $'import subprocess\nimport time\ns = time.time()\nsubprocess.check_call([\'/bin/true\'], close_fds=True)\nprint(time.time() - s)\n'
0.0964419841766 |
|
Date |
User |
Action |
Args |
2019-01-17 04:29:04 | Kirill Kolyshkin | set | recipients:
+ Kirill Kolyshkin |
2019-01-17 04:29:01 | Kirill Kolyshkin | set | messageid: <1547699341.03.0.843639889929.issue35757@roundup.psfhosted.org> |
2019-01-17 04:29:00 | Kirill Kolyshkin | link | issue35757 messages |
2019-01-17 04:29:00 | Kirill Kolyshkin | create | |
|