Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

os.posix_fallocate() generate exception with errno 0 #75289

Closed
socketpair mannequin opened this issue Aug 2, 2017 · 19 comments
Closed

os.posix_fallocate() generate exception with errno 0 #75289

socketpair mannequin opened this issue Aug 2, 2017 · 19 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@socketpair
Copy link
Mannequin

socketpair mannequin commented Aug 2, 2017

BPO 31106
Nosy @vstinner, @larryhastings, @ned-deily, @asvetlov, @socketpair, @serhiy-storchaka, @koobs, @Mariatta, @izbyshev
PRs
  • bpo-31106: Fix handling of erros in posix_fallocate() (#3000) #3000
  • [3.6] bpo-31106: Fix handling of erros in posix_fallocate() and posix… #3088
  • [3.6] bpo-31106: Fix handling of erros in posix_fallocate() and posix_fadvise() (GH-3000) (GH-3000) #4101
    		•
    Files
  • koobs-freebsd-current.non-debug.3.x.build.16.stdio.txt
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-05-24.06:39:10.970>
created_at = <Date 2017-08-02.15:03:06.789>
labels = ['3.7', '3.8', 'type-bug', 'library']
title = 'os.posix_fallocate() generate exception with errno 0'
updated_at = <Date 2018-05-26.22:03:05.681>
user = 'https://github.com/socketpair'

    bugs.python.org fields:

    activity = <Date 2018-05-26.22:03:05.681>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2018-05-24.06:39:10.970>
closer = 'ned.deily'
components = ['Library (Lib)']
creation = <Date 2017-08-02.15:03:06.789>
creator = 'socketpair'
dependencies = []
files = ['47454']
hgrepos = []
issue_num = 31106
keywords = ['patch']
message_count = 19.0
messages = ['299660', '299740', '299741', '299742', '299746', '299747', '299830', '300244', '300259', '300264', '300266', '304876', '308955', '311429', '312453', '312475', '313013', '317538', '317772']
nosy_count = 9.0
nosy_names = ['vstinner', 'larry', 'ned.deily', 'asvetlov', 'socketpair', 'serhiy.storchaka', 'koobs', 'Mariatta', 'izbyshev']
pr_nums = ['3000', '3088', '4101']
priority = None
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31106'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']
    @socketpair
    Copy link
    Mannequin Author

    socketpair mannequin commented Aug 2, 2017

    ===
    os.posix_fallocate(os.open('qwe.qwe', os.O_RDONLY|os.O_CREAT), 0, 1024*1024)
    ===

    generates OSError with errno 0. Suppose this happen due to O_RDONLY flag.

    strace :

    ====
    open("qwe.qwe", O_RDONLY|O_CREAT|O_CLOEXEC, 0777) = 3
    fallocate(3, 0, 0, 1048576) = -1 EBADF (Bad file descriptor)
    ====

    Python 3.5.3, Ubuntu 64-bit.

    @socketpair socketpair mannequin added 3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Aug 2, 2017
    @socketpair
    Copy link
    Mannequin Author

    socketpair mannequin commented Aug 4, 2017

    man posix_fallocate:

    RETURN VALUE

    posix_fallocate() returns zero on success, or an error number on failure. Note that errno is not set.

    @socketpair
    Copy link
    Mannequin Author

    socketpair mannequin commented Aug 4, 2017

    Also, EINTR will not be caught too (!)

    @socketpair
    Copy link
    Mannequin Author

    socketpair mannequin commented Aug 4, 2017

    #3000

    @socketpair
    Copy link
    Mannequin Author

    socketpair mannequin commented Aug 4, 2017

    All checks passed.

    Please merge.

    @socketpair
    Copy link
    Mannequin Author

    socketpair mannequin commented Aug 4, 2017

    posix_fadvise() is also affected :(

    Fixed in this patch.

    @Mariatta
    Copy link
    Member

    Mariatta commented Aug 7, 2017

    Adding Larry who is listed as posix expert.

    @larryhastings
    Copy link
    Contributor

    New changeset d4b93e2 by larryhastings (Коренберг Марк) in branch 'master':
    bpo-31106: Fix handling of erros in posix_fallocate() and posix_fadvise() (bpo-3000) (bpo-3000)
    d4b93e2

    @serhiy-storchaka
    Copy link
    Member

    Needed a NEWS entry.

    @socketpair
    Copy link
    Mannequin Author

    socketpair mannequin commented Aug 14, 2017

    If I understand right, Python 3.5 will not be fixed with this pathc. Right ?

    If yes, I will tell Debian maintainers to backport this patch to Python 3.5, which is shipped with latest stable Debian 9.

    @larryhastings
    Copy link
    Contributor

    You understand correctly. It won't be backported to Python 3.5.

    @serhiy-storchaka
    Copy link
    Member

    Марк, do you mind to add a NEWS entry?

    @socketpair
    Copy link
    Mannequin Author

    socketpair mannequin commented Dec 23, 2017

    Yes, I can. Should I create new PR ? to which branch ?

    @asvetlov
    Copy link
    Contributor

    asvetlov commented Feb 1, 2018

    New changeset 3a04c52 by Andrew Svetlov (Miss Islington (bot)) in branch '3.6':
    bpo-31106: Fix handling of erros in posix_fallocate() and posix_fadvise() (GH-3000) (GH-3000) (bpo-4101)
    3a04c52

    @koobs
    Copy link

    koobs commented Feb 21, 2018

    I believe changeset d4b93e2 may be the cause of buildbot failures on FreeBSD (at least koobs-freebsd-current, log attached), due to only the EBADF error being handled (not EINVAL, et al).

    Unfortunately the worker was been offline for a longer than anticipated period of time and only recently was restored which delayed picking it up.

    The issue (in this workers case) is related to the buildbot home/data directory being on a ZFS filesystem, who's host was recently updated (from current late last year to yesterday).

    Initial investigation/isolation/references:

    Disable posix_fallocate(2) for ZFS
    https://lists.freebsd.org/pipermail/svn-src-head/2017-November/105709.html

    Note: Above change will be relevant (merged) for at least FreeBSD 12 and 11, but perhaps even 10.

    0000684: posix_fallocate() should be allowed to return ENOTSUP (Interp Status: Approved)
    http://austingroupbugs.net/view.php?id=684

    [HEADS UP] posix_fallocate support removed from ZFS, lld affected
    https://lists.freebsd.org/pipermail/freebsd-current/2017-November/067463.html
    Snip: "illumos and ZoL seem to do the same."

    Gold fails when output file is lying on ZFS
    https://sourceware.org/bugzilla/show_bug.cgi?id=22540

    Given the wide scope of use of zfs and both syscalls across multiple OS's, and possible changes to POSIX documentation/standards, changes to the underlying fallocate/fadvise functions may also be indicated here.

    @koobs koobs added the 3.8 only security fixes label Feb 21, 2018
    @izbyshev
    Copy link
    Mannequin

    izbyshev mannequin commented Feb 21, 2018

    The failed test is test_posix_fallocate, not test_posix_fallocate_errno. The former special-cases Solaris for EINVAL already, and the latter merely passes an invalid descriptor.

    It seems that this particular issue is still open only because there is no NEWS entry in the merged commits.

    @ned-deily
    Copy link
    Member

    I agree with Alexey's analysis.

    Koobs, could you please open a new issue about the failing test? Unless someone knows of a foolproof way to determine on all platforms that a file is resident on ZFS, the fix is to skip the test on FreeBSD as well if ZFS is likely to be used there.

    Also, can someone add a NEWS entry and close this issue?

    @ned-deily
    Copy link
    Member

    Considering we have already shipped this fix in 3.6.5, I guess we can live without a NEWS entry at this point -> closing this issue.

    @ned-deily
    Copy link
    Member

    P.S.:

    Koobs, could you please open a new issue about the failing test? Unless someone knows of a foolproof way to determine on all platforms that a file is resident on ZFS, the fix is to skip the test on FreeBSD as well if ZFS is likely to be used there.

    The unrelated problem identified in msg312453 has now been fixed in bpo-33655.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    6 participants
    @asvetlov @koobs @serhiy-storchaka @ned-deily @Mariatta @larryhastings