classification
Title: Misbehavior of BufferedRandom.write with raw file in append mode
Type: behavior Stage: patch review
Components: IO Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, erik.bray, hynek, nitishch, pitrou, serhiy.storchaka, stutzbach
Priority: normal Keywords: patch

Created on 2013-12-27 23:33 by erik.bray, last changed 2018-03-26 12:24 by erik.bray.

Files
File name Uploaded Description Edit
buffered-append-1.patch erik.bray, 2013-12-27 23:33 review
Messages (2)
msg207015 - (view) Author: Erik Bray (erik.bray) * (Python triager) Date: 2013-12-27 23:33
In #18876 I pointed out the following issue:

BufferedWriter/Random doesn't know the raw file was opened with O_APPEND so the writes it shows in the buffer differ from what will actually end up in the file.  For example:

>>> f = open('test', 'wb')
>>> f.write(b'testest')
7
>>> f.close()
>>> f = open('test', 'ab+')
>>> f.tell()
7
>>> f.write(b'A')
1
>>> f.seek(0)
0
>>> f.read()
b'testestA'
>>> f.seek(0)
0
>>> f.read(1)
b't'
>>> f.write(b'B')
1
>>> f.seek(0)
0
>>> f.read()
b'tBstestA'
>>> f.flush()
>>> f.seek(0)
0
>>> f.read()
b'testestAB'

In this example, I read 1 byte from the beginning of the file, then write one byte.  Because of O_APPEND, the effect of the write() call on the raw file is to append, regardless of where BufferedWriter seeks it to first.  But before the f.flush() call f.read() just shows what's in the buffer which is not what will actually be written to the file.  (Naturally, unbuffered io does not have this particular problem.)


Now that #18876 we can test if a file was opened in append mode and correct for this.  The attach patch includes a pretty simple solution that manually calls buffered_seek at the beginning of bufferedwriter_write if the raw file is in append mode.  In doing so it made sense to split buffered_seek into two separate functions.

This might be overkill, however.
msg314448 - (view) Author: Erik Bray (erik.bray) * (Python triager) Date: 2018-03-26 12:24
I keep forgetting about this (given that it's like 5 years old now).  Let me see if I can make a new PR for it...
History
Date User Action Args
2018-03-26 12:24:47erik.braysetmessages: + msg314448
2018-02-02 06:53:56nitishchsetnosy: + nitishch
2014-11-18 17:27:26serhiy.storchakasetnosy: + pitrou, benjamin.peterson, stutzbach, hynek, serhiy.storchaka
stage: patch review

versions: + Python 2.7, Python 3.4, Python 3.5
2013-12-27 23:33:12erik.braycreate