classification
Title: unsynchronized write pointer in io.TextIOWrapper in 'r+' mode
Type: behavior Stage:
Components: IO, Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder: TextIOWrapper: issues with interlaced read-write
View: 12215
Assigned To: Nosy List: Manuel Ignacio Pérez Alcolea, josh.r, martin.panter
Priority: normal Keywords:

Created on 2019-11-06 04:04 by Manuel Ignacio Pérez Alcolea, last changed 2019-11-17 23:20 by martin.panter.

Messages (2)
msg356089 - (view) Author: Manuel Ignacio Pérez Alcolea (Manuel Ignacio Pérez Alcolea) Date: 2019-11-06 04:04
There seems to be a bug in the `io.TextIOWrapper` class while working in 'r+' mode, although I can't say the source of the problem is right there.

The write pointer doesn't match `file.tell()` after performing a read operation.

For example, this file, consisting of 3 lines:

  line one
  line two
  line three

Doesn't result in the expected modification running the following program:

with open('file', 'r+', buffering=1) as f:
    print(f.tell())                  # => 0
    print(f.readline().strip())      # we read 1 line
    print(f.tell())                  # => 9  
    print('Hello', file=f)           # we write "Hello\n"
    print(f.tell())                  # => 34

Instad of

  line one
  Hello
  wo
  line three

It results in

  line one
  line two
  line threeHello

But it works just fine if `f.seek(f.tell())` is added the program, right before the write operation.

There are several possible explanations on StackOverflow, involving the buffering for IO in text files:

https://stackoverflow.com/a/58722058/11601118
msg356835 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2019-11-17 23:20
Previously Issue 12215 and a couple of other duplicates were opened about this. Writing after reading with TextIOWrapper doesn't work as people expect. The report was closed apparently because Victor thought there wasn't enough interest in it.

FWIW the seek-then-write workaround will probably work for most common codecs, but would suffer the same problems discussed in Issue 26158 for codecs like UTF-7 and ISO-2022 that would need the encoder state constructed from the decoder state.
History
Date User Action Args
2019-11-17 23:20:53martin.pantersetsuperseder: TextIOWrapper: issues with interlaced read-write

messages: + msg356835
nosy: + martin.panter
2019-11-07 00:36:28josh.rsetnosy: + josh.r
2019-11-07 00:29:02josh.rsetcomponents: + Library (Lib)
2019-11-06 04:04:15Manuel Ignacio Pérez Alcoleacreate