classification
Title: Optimization: use close_range(2) if available
Type: enhancement Stage: resolved
Components: C API Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: gregory.p.smith, kevans, kevans91, miss-islington
Priority: normal Keywords: patch

Created on 2020-04-28 14:10 by kevans91, last changed 2020-10-19 20:38 by gregory.p.smith. This issue is now closed.

Files
File name Uploaded Description Edit
cpython-close_range.diff kevans91, 2020-04-28 14:10
Pull Requests
URL Status Linked Edit
PR 22651 merged kevans, 2020-10-11 19:13
Messages (6)
msg367531 - (view) Author: Kyle Evans (kevans91) * Date: 2020-04-28 14:10
This is dependent on issue40422; the diff on top of that (PR19075) looks like the attached. Effectively, close_range(2) should be preferred at all times if it's available, otherwise we'll use closefrom(2) if available with a fallback to fdwalk(3) or plain old loop over fd range in order of most efficient to least.

PR will be sent after issue40422 is resolved.
msg378445 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2020-10-11 18:48
for reference, very recent Linux kernels appear to have gained a close_range syscall.  http://lkml.iu.edu/hypermail/linux/kernel/2008.0/02649.html

Your diff isn't quite sufficient as is.  When depending on a syscall that has a function provided by libc, the libc function may exist (thus HAVE_CLOSE_RANGE will be true at Python compile time) but the system the process is running on may not support the system call.  So it'll return an EINVAL (or something like that) error.

Special handling of that error to add an `else {...}` falling back to the other codepath is necessary.
msg378448 - (view) Author: Kyle Evans (kevans) Date: 2020-10-11 18:57
Ah, I will fix this and then submit a PR, thanks... hopefully it returns ENOSYS.
msg378449 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2020-10-11 19:16
Ah, yeah ENOSYS is it.  I had to do this trick in older subprocess versions for something else.  Still visible here in the old 2.7 backport: https://github.com/google/python-subprocess32/blob/main/_posixsubprocess.c#L801
msg378452 - (view) Author: miss-islington (miss-islington) Date: 2020-10-11 20:18
New changeset 1800c600801709958af66bebfa683a4e7570810f by Kyle Evans in branch 'master':
bpo-40423: Optimization: use close_range(2) if available (GH-22651)
https://github.com/python/cpython/commit/1800c600801709958af66bebfa683a4e7570810f
msg379011 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2020-10-19 20:38
thanks Kyle!
History
Date User Action Args
2020-10-19 20:38:17gregory.p.smithsetstatus: open -> closed
resolution: fixed
messages: + msg379011

stage: patch review -> resolved
2020-10-11 20:18:56miss-islingtonsetnosy: + miss-islington
messages: + msg378452
2020-10-11 19:21:38gregory.p.smithsetassignee: gregory.p.smith
2020-10-11 19:16:39gregory.p.smithsetmessages: + msg378449
2020-10-11 19:13:46kevanssetstage: patch review
pull_requests: + pull_request21624
2020-10-11 18:57:46kevanssetnosy: + kevans
messages: + msg378448
2020-10-11 18:48:27gregory.p.smithsetversions: + Python 3.10, - Python 3.9
2020-10-11 18:48:19gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg378445
2020-04-28 14:10:08kevans91create