This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: fall back os.fdatasync() to fsync() on POSIX systems without fdatasync() support
Type: enhancement Stage: resolved
Components: IO Versions:
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: cebtenzzre, gmelikov, gregory.p.smith, ronaldoussoren, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2020-02-15 19:06 by gmelikov, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 18516 open gmelikov, 2020-02-15 19:25
Messages (6)
msg362025 - (view) Author: George Melikov (gmelikov) * Date: 2020-02-15 19:06
POSIX fdatasync() is similar to fsync() but it tries not to sync non-needed metadata. If POSIX OS doesn't have it - it's safe to use fsync() (If we need to sync data to disk - we have to use one of these functions).

This change will help to run code with fdatasync() on MacOS without fallbacks in Python code.

I'll propose a PR soon.
msg362027 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-02-15 19:35
fsync() may be slower than fdatasync(). There may be cases in which you prefer to not call fsync() or call it less often if POSIX fdatasync() is not available. To do this you should know whether Python fdatasync() calls POSIX fdatasync() or fsync(). The simplest way to provide this information to user is to not provide os.fdatasync() in the underlying system call is not available.

Note also that this change will not help to run code with fdatasync() on MacOS without fallbacks in Python code until you drop support of all Python versions older than 3.9.
msg362028 - (view) Author: George Melikov (gmelikov) * Date: 2020-02-15 19:42
If there is a way not to sync data - you should use neither fdatasync nor fsync.

So IMHO if someone wants to sync data and want to use fdatasync - I see the only way on MacOS is to use fsync().

> Note also that this change will not help to run code with fdatasync() on MacOS without fallbacks in Python code until you drop support of all Python versions older than 3.9.

I agree, but I propose to make this change in 3.9 to get it somewhere in future.
msg362029 - (view) Author: George Melikov (gmelikov) * Date: 2020-02-15 19:44
I want to add that it's a usual practice:
- https://github.com/libuv/libuv/pull/1580/files/bdd987a9b4347164e31e22d2d5ce06fbb5ebc859
- https://rev.ng/gitlab/revng/qemu/commit/6f1953c4c14566d3303709869fd26201828b3ccf
msg375824 - (view) Author: George Melikov (gmelikov) * Date: 2020-08-23 19:31
PR rebased and ready to review.
msg415587 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2022-03-20 01:17
The os module provides a pretty low level simple shim over platform APIs. It is better for logic like this to live in a higher level application library rather than make big assumptions on the part of the user.

```
try:
    os.fdatasync(fd)
except Exception as err:
    logging.debug("fdatasync(fd) failed %s, falling back to fsync(fd)", err)
    os.fsync(fd)
```
History
Date User Action Args
2022-04-11 14:59:26adminsetgithub: 83821
2022-03-20 01:17:54gregory.p.smithsetstatus: open -> closed

type: enhancement

nosy: + gregory.p.smith
messages: + msg415587
resolution: rejected
stage: patch review -> resolved
2022-03-19 20:40:36cebtenzzresetnosy: + cebtenzzre
2020-10-20 15:30:53ned.deilysetnosy: + ronaldoussoren
2020-08-23 19:31:09gmelikovsetmessages: + msg375824
2020-02-15 19:44:24gmelikovsetmessages: + msg362029
2020-02-15 19:42:16gmelikovsetmessages: + msg362028
2020-02-15 19:35:40serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg362027
2020-02-15 19:25:47gmelikovsetkeywords: + patch
stage: patch review
pull_requests: + pull_request17893
2020-02-15 19:06:08gmelikovcreate