This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Misbehavior of BufferedRandom.write with raw file in append mode
Type: behavior Stage: patch review
Components: IO Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, erik.bray, hynek, nitishch, pitrou, serhiy.storchaka, stutzbach
Priority: normal Keywords: patch

Created on 2013-12-27 23:33 by erik.bray, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
buffered-append-1.patch erik.bray, 2013-12-27 23:33 review
Pull Requests
URL Status Linked Edit
PR 21729 open erik.bray, 2020-08-04 11:02
Messages (2)
msg207015 - (view) Author: Erik Bray (erik.bray) * (Python triager) Date: 2013-12-27 23:33
In #18876 I pointed out the following issue:

BufferedWriter/Random doesn't know the raw file was opened with O_APPEND so the writes it shows in the buffer differ from what will actually end up in the file.  For example:

>>> f = open('test', 'wb')
>>> f.write(b'testest')
7
>>> f.close()
>>> f = open('test', 'ab+')
>>> f.tell()
7
>>> f.write(b'A')
1
>>> f.seek(0)
0
>>> f.read()
b'testestA'
>>> f.seek(0)
0
>>> f.read(1)
b't'
>>> f.write(b'B')
1
>>> f.seek(0)
0
>>> f.read()
b'tBstestA'
>>> f.flush()
>>> f.seek(0)
0
>>> f.read()
b'testestAB'

In this example, I read 1 byte from the beginning of the file, then write one byte.  Because of O_APPEND, the effect of the write() call on the raw file is to append, regardless of where BufferedWriter seeks it to first.  But before the f.flush() call f.read() just shows what's in the buffer which is not what will actually be written to the file.  (Naturally, unbuffered io does not have this particular problem.)


Now that #18876 we can test if a file was opened in append mode and correct for this.  The attach patch includes a pretty simple solution that manually calls buffered_seek at the beginning of bufferedwriter_write if the raw file is in append mode.  In doing so it made sense to split buffered_seek into two separate functions.

This might be overkill, however.
msg314448 - (view) Author: Erik Bray (erik.bray) * (Python triager) Date: 2018-03-26 12:24
I keep forgetting about this (given that it's like 5 years old now).  Let me see if I can make a new PR for it...
History
Date User Action Args
2022-04-11 14:57:56adminsetgithub: 64281
2020-08-04 11:02:17erik.braysetpull_requests: + pull_request20873
2018-03-26 12:24:47erik.braysetmessages: + msg314448
2018-02-02 06:53:56nitishchsetnosy: + nitishch
2014-11-18 17:27:26serhiy.storchakasetnosy: + pitrou, benjamin.peterson, stutzbach, hynek, serhiy.storchaka
stage: patch review

versions: + Python 2.7, Python 3.4, Python 3.5
2013-12-27 23:33:12erik.braycreate