classification
Title: Open file in a+ mode reads from end of file in Python 3.4
Type: behavior Stage:
Components: Documentation Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Arfrever, caitlinwalker37, docs@python, eryksun, georg.brandl, nicksjacobson, pitrou, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords:

Created on 2014-10-16 07:57 by nicksjacobson, last changed 2017-01-26 08:08 by vstinner. This issue is now closed.

Messages (10)
msg229511 - (view) Author: Nick Jacobson (nicksjacobson) Date: 2014-10-16 07:57
In Python 2.7.8.10 running the following gives one result:

    >>> with open(r"C:\myfile.txt", "a+") as f:
    ...     f.tell()
    ...
    0L

But in Python 3.4.1.0 it gives a different result:

    >>> with open(r"C:\myfile.txt", "a+") as f:
    ...     f.tell()
    ...
    98

According to the man page for fopen, for a+ mode: "The initial file position for reading is at the beginning of the file." Source: http://linux.die.net/man/3/fopen
msg229512 - (view) Author: Nick Jacobson (nicksjacobson) Date: 2014-10-16 07:57
Note: I'm running this in Windows 7, same result on Windows Server 2008.
msg229513 - (view) Author: Nick Jacobson (nicksjacobson) Date: 2014-10-16 07:59
I also should have mentioned that C:\myfile.txt in my example is 98 bytes long, so it is being read from the end instead of the beginning.
msg229514 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-10-16 08:03
Python 3 documentation is explicit about the "a" mode:
"open for writing, appending *to the end* of the file if it exists"
https://docs.python.org/dev/library/functions.html#open

Python 2 is based on the stdio.h of the C standard library which behaves differently. fopen() documentation:
"Open for appending (writing *at end of file*).
http://linux.die.net/man/3/fopen

IMO it's a bug in the C library. It's probably yet another bug in the C standard library on Windows:
http://haypo-notes.readthedocs.org/python.html#bugs-in-the-c-stdio-used-by-the-python-i-o
msg229515 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-10-16 08:11
Oh, fopen() behaves differently between "a" and "a+" mode on Linux: "a" goes to the end, "a+" doesn't. It looks like a bug in the C library, or a bug in the documentation?
"a+ mode:
Open for reading and appending (writing at end of file)"
http://linux.die.net/man/3/fopen

open() of Python 3 and io.open() of Python 2 and Python 3 behaves "correctly": "a" and "a+" modes go to the end.
msg229516 - (view) Author: Nick Jacobson (nicksjacobson) Date: 2014-10-16 08:19
For writing, end of file seems correct. But for reading, the documentation says it should be the beginning of the file.

"The initial file position for reading is at the beginning of the file, but output is always appended to the end of the file." http://linux.die.net/man/3/fopen
msg229517 - (view) Author: Eryk Sun (eryksun) * Date: 2014-10-16 08:53
FYI, this is implementation defined in C89:

    ... unless the file is opened with append mode in which case 
    it is **implementation-defined** whether the file position 
    indicator is positioned at the beginning or the end of the 
    file.

http://port70.net/~nsz/c/c89/c89-draft.html#4.9.3
msg229518 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-10-16 08:55
This is a somewhat unfortunate difference between the major C libs and Python's new IO, but probably too late to change.

What's left to do is to document the behavior properly.
msg229519 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-10-16 09:29
> This is a somewhat unfortunate difference between the major C libs and Python's new IO, but probably too late to change.

Well, it's easy to workaround it: just all file.seek(0, 2) after open() to always to the end ;-) (or file.seek(0) to always go the beginning).
msg286296 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-01-26 08:08
On Python 2, it's easy to workaround the issue, just always go the end explicitly:

   f = open(filename, "a+")
   f.seek(0, os.SEEK_END)

Or use the io module which doesn't use the stdio of the C library and has a portable and reliable behaviour:

   f = io.open(filename, "a+")


Georg Brandl: "This is a somewhat unfortunate difference between the major C libs and Python's new IO, but probably too late to change."

Since it's easy to workaround the issue and only one user complained lat 2 years (3 years?), sorry, I close now the issue.

I add this bug to my long list of bugs in the C stdio which won't be fixed in Python 2...
http://haypo-notes.readthedocs.io/python.html#bugs-in-the-c-stdio-used-by-the-python-i-o
History
Date User Action Args
2017-01-26 08:08:35vstinnersetstatus: open -> closed
resolution: wont fix
messages: + msg286296
2017-01-26 07:56:08vstinnersetmessages: - msg286294
2017-01-26 06:39:29caitlinwalker37setnosy: + caitlinwalker37
messages: + msg286294
2014-10-17 03:19:19eryksunsetversions: + Python 3.4, Python 3.5, - Python 2.7
title: Python 2: Open file in a+ mode doesn't go to the end -> Open file in a+ mode reads from end of file in Python 3.4
2014-10-16 12:38:41Arfreversetnosy: + Arfrever
title: Python 2: Open file in a+ mode on Windows doesn't go to the end -> Python 2: Open file in a+ mode doesn't go to the end

versions: + Python 2.7, - Python 3.4
2014-10-16 09:29:22vstinnersetmessages: + msg229519
2014-10-16 08:55:55georg.brandlsetnosy: + georg.brandl, docs@python
messages: + msg229518

assignee: docs@python
components: + Documentation, - Windows, IO
2014-10-16 08:53:04eryksunsetnosy: + eryksun
messages: + msg229517
2014-10-16 08:19:06nicksjacobsonsetmessages: + msg229516
2014-10-16 08:11:49vstinnersetmessages: + msg229515
2014-10-16 08:03:34vstinnersettitle: Open file in a+ mode reads from end of file in Python 3.4 -> Python 2: Open file in a+ mode on Windows doesn't go to the end
nosy: + pitrou, tim.golden, vstinner, zach.ware, steve.dower

messages: + msg229514

components: + Windows
2014-10-16 07:59:21nicksjacobsonsetmessages: + msg229513
2014-10-16 07:57:46nicksjacobsonsetmessages: + msg229512
2014-10-16 07:57:11nicksjacobsoncreate