This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: mmap requires file to be synced
Type: behavior Stage: resolved
Components: None Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rosslagerwall Nosy List: python-dev, rion4ik@gmail.com, rosslagerwall, schmir, sdaoden, vstinner
Priority: normal Keywords: patch

Created on 2011-05-18 09:56 by rion4ik@gmail.com, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
12102.2.diff sdaoden, 2011-05-21 12:43 review
Messages (14)
msg136217 - (view) Author: rion (rion4ik@gmail.com) Date: 2011-05-18 09:56
Not flushed file gives next results:

rion@rionhost ~/temp $ python2 test.py 
Traceback (most recent call last):
  File "test.py", line 10, in <module>
    m = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
mmap.error: [Errno 22] Invalid argument
rion@rionhost ~/temp $ python3 test.py 
Traceback (most recent call last):
  File "test.py", line 10, in <module>
    m = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
ValueError: mmap offset is greater than file size


testcase:

import mmap
from tempfile import TemporaryFile

f = TemporaryFile()
f.write(b"hello world")

#f.flush()

m = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)


just uncomment f.flush() and try again. feel the difference.

suggestion: force flush on file descriptor when passed to mmap.
msg136218 - (view) Author: Ralf Schmitt (schmir) Date: 2011-05-18 10:21
your suggestion doesn't work. there's no such thing like a "flush on file descriptor".
msg136220 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-05-18 10:23
See also issue #11877 and #11277.
msg136221 - (view) Author: rion (rion4ik@gmail.com) Date: 2011-05-18 10:39
okay guys I'm not going to read all this stuff.
just document it or fix.

thanks for quick reply
msg136236 - (view) Author: Steffen Daode Nurpmeso (sdaoden) Date: 2011-05-18 12:31
@ rion wrote (2011-05-18 12:39+0200):
> just document it or fix.

Hello, zion, Victor, i'm proposing a documentation patch.
It applies to 2.7 and 3.3 (from yesterday).
msg136237 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-05-18 12:33
> Hello, zion, Victor, i'm proposing a documentation patch.

I like it.

I don't think that Python should guess what the user expects (i.e. Python should not sync the file *implicitly*).
msg136242 - (view) Author: Steffen Daode Nurpmeso (sdaoden) Date: 2011-05-18 13:09
@ STINNER Victor wrote (2011-05-18 14:33+0200):
> I don't think that Python should guess what the user expects
> (i.e. Python should not sync the file *implicitly*).

Before i've found F_FULLFSYNC i indeed have had a solution which
tracked the open() of all files, so that mmap() could check wether
a file had been opened in write mode or not.  ;|
Python does not do that by default (AFAIK).
msg136243 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-05-18 13:23
> I don't think that Python should guess what the user expects (i.e. Python should not sync the file *implicitly*).

Agreed.

The documentation patch looks good.
msg136434 - (view) Author: Steffen Daode Nurpmeso (sdaoden) Date: 2011-05-21 12:43
Looked at it again and i think it's much better english with an
additional ..to ensure "that" local...
@Ross, aren't you a native english speaker?  What do you say?
msg136452 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-05-21 15:35
> Looked at it again and i think it's much better english with an
> additional ..to ensure "that" local...
> @Ross, aren't you a native english speaker?  What do you say?

Yes I am, but that doesn't make me any good ;-)

I'd probably agree that it's better with "that" included.
msg136545 - (view) Author: Steffen Daode Nurpmeso (sdaoden) Date: 2011-05-22 17:32
> that doesn't make me any good

Well - 'can only be better than myself, so i'll take that as yes :)
msg141069 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-07-25 05:15
New changeset 198627bbe242 by Ross Lagerwall in branch '3.2':
Issue #12102: Document that buffered files must be flushed before being used
http://hg.python.org/cpython/rev/198627bbe242

New changeset 4898b14dcd69 by Ross Lagerwall in branch 'default':
Issue #12102: Merge with 3.2.
http://hg.python.org/cpython/rev/4898b14dcd69
msg141070 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-07-25 05:25
New changeset e75715f27ce7 by Ross Lagerwall in branch '2.7':
Issue #12102: Document that buffered files must be flushed before being used
http://hg.python.org/cpython/rev/e75715f27ce7
msg141071 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-07-25 05:26
Thanks!
History
Date User Action Args
2022-04-11 14:57:17adminsetgithub: 56311
2011-07-25 05:26:23rosslagerwallsetstatus: open -> closed
versions: + Python 3.3
messages: + msg141071

assignee: rosslagerwall
resolution: fixed
stage: resolved
2011-07-25 05:25:13python-devsetmessages: + msg141070
2011-07-25 05:15:53python-devsetnosy: + python-dev
messages: + msg141069
2011-05-22 17:32:18sdaodensetfiles: - 12102.1.diff
2011-05-22 17:32:14sdaodensetmessages: + msg136545
2011-05-21 15:35:47rosslagerwallsetmessages: + msg136452
2011-05-21 12:43:31sdaodensetfiles: + 12102.2.diff

messages: + msg136434
2011-05-18 13:23:14rosslagerwallsetnosy: + rosslagerwall
messages: + msg136243
2011-05-18 13:09:54sdaodensetmessages: + msg136242
2011-05-18 12:33:33vstinnersetmessages: + msg136237
2011-05-18 12:31:01sdaodensetfiles: + 12102.1.diff
keywords: + patch
messages: + msg136236
2011-05-18 10:39:42rion4ik@gmail.comsetmessages: + msg136221
2011-05-18 10:23:20vstinnersetnosy: + vstinner, sdaoden
messages: + msg136220
2011-05-18 10:21:17schmirsetnosy: + schmir
messages: + msg136218
2011-05-18 09:56:03rion4ik@gmail.comcreate