This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: file opened for updating cannot write after read
Type: behavior Stage: resolved
Components: IO, Windows Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, jkloth, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2017-05-24 19:07 by jkloth, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (3)
msg294379 - (view) Author: Jeremy Kloth (jkloth) * Date: 2017-05-24 19:07
Attempting to append to an existing file fails with no error set:

>>> import os, tempfile
>>> fd, fn = tempfile.mkstemp()
>>> os.write(fd, 'some text')
9
>>> os.close(fd)
>>> with open(fn, 'r+') as f:
...     f.read()
...     f.write('more text')
...     
'some text'
Traceback (most recent call last):
  File "<interactive input>", line 3, in <module>
IOError: [Errno 0] Error

(error 0 is defined as NO_ERROR)
msg294414 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2017-05-24 23:36
It might be simplest to close this as 3rd party since it's similar to bpo-29817. A workaround is to call f.seek(0, 1) to reset the stream state when switching between reading and writing. 

That said, a switch to writing at EOF should be supported. The problem is that Python 2's file_read function in fileobject.c executes the following code in this case:

        if (bytesread < buffersize && !interrupted) {
            clearerr(f->f_fp);
            break;
        }

clearerr() resets the EOF flag on the stream, so the Windows CRT has no way to know that it's allowed to reset the buffer from read-mode to write-mode in the subsequent write. Finally, Python raises a weird 'success' exception because the CRT writes fewer than the number of requested bytes without setting errno. AFAIK, this scenario only occurs on Windows. glibc on Linux appears to be more robust in this case.

A simple way to demonstrate this without involving a debugger is to add a second read() call before the write(). The 2nd read() takes a different path in file_read(), which doesn't clear the stream's EOF flag. 

    import os, tempfile
    fd, fn = tempfile.mkstemp()
    os.write(fd, 'some text')
    os.close(fd)
    f = open(fn, 'r+')

    >>> f.read()
    'some text'
    >>> f.read()
    ''
    >>> f.write('more text')
    >>> f.close()

    >>> open(fn).read()
    'some textmore text'
msg294509 - (view) Author: Jeremy Kloth (jkloth) * Date: 2017-05-25 19:28
It seems to me that it is a quite simple fix:

--- a/Objects/fileobject.c
+++ b/Objects/fileobject.c
@@ -1110,7 +1110,7 @@ file_read(PyFileObject *f, PyObject *args)
-            clearerr(f->f_fp);
+            if (ferror(f->f_fp)) clearerr(f->f_fp);

which is exactly what the empty read case does above this.

This fixes my error, and produces no changes to the test suite.

I guess it could be wrapped in an #ifdef MS_WINDOWS, just it case, but that seems unnecessary as the previous code does not.
History
Date User Action Args
2022-04-11 14:58:46adminsetgithub: 74645
2021-02-25 17:32:06eryksunsetstatus: open -> closed
resolution: out of date
stage: resolved
2017-05-25 19:28:44jklothsetmessages: + msg294509
2017-05-24 23:36:18eryksunsettype: behavior

messages: + msg294414
nosy: + eryksun
2017-05-24 19:07:32jklothcreate