Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use fcntl(fd, F_GETFD) to check whether an fd is valid #90073

Closed
tiran opened this issue Nov 28, 2021 · 6 comments
Closed

Use fcntl(fd, F_GETFD) to check whether an fd is valid #90073

tiran opened this issue Nov 28, 2021 · 6 comments
Labels
3.11 only security fixes topic-C-API type-bug An unexpected behavior, bug, or error

Comments

@tiran
Copy link
Member

tiran commented Nov 28, 2021

BPO 45915
Nosy @vstinner, @tiran, @eryksun, @corona10
PRs
  • bpo-45915: use fcntl(fd, F_GETFD) in is_valid_fd() (GH-29821) #29821
  • bpo-45919: Use WinAPI GetFileType() in is_valid_fd() #30082
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2021-11-28.10:13:06.935>
labels = ['expert-C-API', 'type-bug', '3.11']
title = 'Use fcntl(fd, F_GETFD) to check whether an fd is valid'
updated_at = <Date 2021-12-14.03:31:17.636>
user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2021-12-14.03:31:17.636>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['C API']
creation = <Date 2021-11-28.10:13:06.935>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 45915
keywords = ['patch']
message_count = 5.0
messages = ['407197', '407217', '407222', '407224', '408514']
nosy_count = 4.0
nosy_names = ['vstinner', 'christian.heimes', 'eryksun', 'corona10']
pr_nums = ['29821', '30082']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45915'
versions = ['Python 3.11']
    @tiran
    Copy link
    Member Author

    tiran commented Nov 28, 2021

    is_valid_fd() uses dup() or fstat() to check whether an fd is valid. Both operations are costly.

    fcntl() with F_GETFD returns the file descriptor flags (e.g. CLOEXEC) and -1 with errno set to EBADF when fd is not an open file descriptor. It's faster than duplicating + closing a fd or calling fstat(). The idea to use fcntl(fd, F_GETFD) is inspired by the patch [1]. According to Stackoverflow [2]:

    fcntl(fd, F_GETFD) is the canonical cheapest way to check that fd is a valid open file descriptor.
    F_GETFD is cheaper in principle since it only dereferences the (process-local) file descriptor in kernel space, not the underlying open file description (process-shared) which it refers to.

    [1] singlestore-labs@0364554

    [2] https://stackoverflow.com/questions/12340695/how-to-check-if-a-given-file-descriptor-stored-in-a-variable-is-still-valid

    @tiran tiran added 3.11 only security fixes topic-C-API type-bug An unexpected behavior, bug, or error labels Nov 28, 2021
    @tiran
    Copy link
    Member Author

    tiran commented Nov 28, 2021

    New changeset f87ea03 by Christian Heimes in branch 'main':
    bpo-45915: use fcntl(fd, F_GETFD) in is_valid_fd() (GH-29821)
    f87ea03

    @eryksun
    Copy link
    Contributor

    eryksun commented Nov 28, 2021

    I've created bpo-45919 with a suggested enhancement to use GetFileType() in Windows, since the Windows C runtime does not provide fcntl().

    @tiran
    Copy link
    Member Author

    tiran commented Nov 28, 2021

    Thank you!

    @eryksun
    Copy link
    Contributor

    eryksun commented Dec 14, 2021

    PR 29821 adds __APPLE__ to the platforms that use fcntl(fd, F_GETFD). Is this okay on macOS, given bpo-30225? Apparently fstat() fails if the other end of a pipe is closed.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @kumaraditya303
    Copy link
    Contributor

    Fixed by #29821

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.11 only security fixes topic-C-API type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @tiran @eryksun @kumaraditya303