classification
Title: mmap.flush() on Linux does not accept the "offset" and "size" args
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: berker.peksag, byronhawkins, christian.heimes, docs@python, miss-islington, pablogsal, socketpair, vstinner, xiang.zhang
Priority: low Keywords: easy, patch

Created on 2018-02-08 15:06 by byronhawkins, last changed 2018-10-20 00:52 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 5621 merged pablogsal, 2018-02-11 15:21
PR 9989 merged miss-islington, 2018-10-20 00:38
PR 9990 merged miss-islington, 2018-10-20 00:38
PR 9991 merged miss-islington, 2018-10-20 00:39
Messages (12)
msg311832 - (view) Author: Byron Hawkins (byronhawkins) Date: 2018-02-08 15:06
open_file = open("file.txt", "r+b")
file_map = mmap.mmap(open_file, 0)
file_map.seek(offset)
file_map.write("foobar") # success
file_map.flush() # success
file_map.flush(offset, len("foobar")) # Fails with "errno 22"

This example passes correct arguments to mmap.flush(), yet it fails with errno 22. So the arguments are not valid on linux. If the bug cannot be fixed, then all reference to those two arguments should be removed from the code and documentation related to Linux.
msg311833 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-02-08 15:11
offset must be a multiple of pagesize. This is a known but undocumented restriction on some platforms, see http://man7.org/linux/man-pages/man2/msync.2.html
msg311834 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-02-08 15:14
I'm marking this as a low priority and simple documentation bug. The mmap module exposes the page size of the architecture as mmap.PAGESIZE.
msg311835 - (view) Author: Byron Hawkins (byronhawkins) Date: 2018-02-08 15:22
Couldn't the implementation check the page size and throw a better error (for relevant platforms)? Or better yet, adjust the offset to the nearest inclusive page boundary.
msg311882 - (view) Author: Марк Коренберг (socketpair) * Date: 2018-02-09 12:48
Actually documented: 

http://man7.org/linux/man-pages/man2/msync.2.html

EINVAL: addr is not a multiple of PAGESIZE;
msg312098 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2018-02-13 03:32
One thing to note is the behaviour seems fit implementation detail. POSIX doesn't requires this for both mmap and msync, it's optional:

The mmap( ) function may fail if:
[EINVAL] The addr argument (if MAP_FIXED was specified) or off is not a multiple of the page size as returned

The msync( ) function may fail if:
[EINVAL] The value of addr is not a multiple of the page size as returned by sysconf( ).
msg314626 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2018-03-29 00:32
By the way, there is a note in the "Change History" section of the msync() documentation at http://pubs.opengroup.org/onlinepubs/9699919799/

    The second [EINVAL] error condition is made mandatory.

The second EINVAL error condition they mention is:

    [EINVAL] The value of addr is not a multiple of the page size as returned by sysconf().

So unless I'm missing something obvious, the "the msync() function may fail if" part may now be outdated?

Also, I noticed that we don't have tests for mm.flush(offset) and mm.flush(offset, size) cases. It would be nice to add some tests even if we decide to not merge the documentation PR.
msg328103 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-20 00:38
New changeset 027664a3d5ebad575aafe5fcc572e3b05f7f24e5 by Victor Stinner (Pablo Galindo) in branch 'master':
bpo-32798: Add restriction on the offset parameter for mmap.flush in the docs (#5621)
https://github.com/python/cpython/commit/027664a3d5ebad575aafe5fcc572e3b05f7f24e5
msg328111 - (view) Author: miss-islington (miss-islington) Date: 2018-10-20 00:47
New changeset 75ee130c39e73730535d94923fd8322ef616cb83 by Miss Islington (bot) in branch '3.6':
bpo-32798: Add restriction on the offset parameter for mmap.flush in the docs (GH-5621)
https://github.com/python/cpython/commit/75ee130c39e73730535d94923fd8322ef616cb83
msg328112 - (view) Author: miss-islington (miss-islington) Date: 2018-10-20 00:48
New changeset 557a68789b97bf281aa7b7e96f083982c01a5f7e by Miss Islington (bot) in branch '3.7':
bpo-32798: Add restriction on the offset parameter for mmap.flush in the docs (GH-5621)
https://github.com/python/cpython/commit/557a68789b97bf281aa7b7e96f083982c01a5f7e
msg328114 - (view) Author: miss-islington (miss-islington) Date: 2018-10-20 00:49
New changeset 2bad7acdfebb87a6eef238a7acca636cfb648a02 by Miss Islington (bot) in branch '2.7':
bpo-32798: Add restriction on the offset parameter for mmap.flush in the docs (GH-5621)
https://github.com/python/cpython/commit/2bad7acdfebb87a6eef238a7acca636cfb648a02
msg328116 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-20 00:52
The documentation has been clarified, thanks Pablo Galindo!
History
Date User Action Args
2018-10-20 00:52:12vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg328116

stage: patch review -> resolved
2018-10-20 00:49:42miss-islingtonsetmessages: + msg328114
2018-10-20 00:48:09miss-islingtonsetmessages: + msg328112
2018-10-20 00:47:19miss-islingtonsetnosy: + miss-islington
messages: + msg328111
2018-10-20 00:39:55miss-islingtonsetpull_requests: + pull_request9332
2018-10-20 00:38:17miss-islingtonsetpull_requests: + pull_request9331
2018-10-20 00:38:09miss-islingtonsetpull_requests: + pull_request9330
2018-10-20 00:38:00vstinnersetnosy: + vstinner
messages: + msg328103
2018-03-29 00:32:01berker.peksagsetmessages: + msg314626
2018-02-13 03:32:56xiang.zhangsetnosy: + pablogsal, berker.peksag, xiang.zhang
messages: + msg312098
2018-02-11 15:21:46pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request5430
2018-02-09 12:48:23socketpairsetnosy: + socketpair
messages: + msg311882
2018-02-08 15:22:34byronhawkinssetmessages: + msg311835
2018-02-08 15:14:12christian.heimessetkeywords: + easy

messages: + msg311834
2018-02-08 15:11:03christian.heimessetpriority: normal -> low

type: crash -> behavior
assignee: docs@python
components: + Documentation, - Library (Lib)
versions: + Python 3.6, Python 3.7, Python 3.8
nosy: + docs@python, christian.heimes

messages: + msg311833
2018-02-08 15:06:58byronhawkinscreate