classification
Title: f.tell() returning negative number on Windows build
Type: behavior Stage: resolved
Components: Documentation, IO Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: docs@python Nosy List: bakuriu, benjamin.peterson, catalin.iacob, docs@python, pitrou, python-dev, r.david.murray, rhettinger, serhiy.storchaka, sijinjoseph, skrah
Priority: normal Keywords: patch

Created on 2012-10-18 02:08 by rhettinger, last changed 2013-07-30 19:56 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
16273-doc-update.patch sijinjoseph, 2013-04-13 20:15 review
Messages (14)
msg173226 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2012-10-18 02:08
On a fresh 64-bit install of Windows 2.7.3 running on Windows 7, f.tell() is producing unexpected (and unusable) results:

>>> f = open('sample.txt')
>>> f.read(3)
>>> f.read(3)
>>> f.tell()
-5L
msg173236 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-10-18 07:39
What tell() returns before any reading? What about non-buffered files?
msg180145 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2013-01-17 17:39
FWIW, on Windows only, "open('foo', 'a').tell()" always returns 0, which I think is broken too.  It usually works anyway, because it updates the position before the first .write().  But it does not work in case the writing occurs in another process, like described here:

http://stackoverflow.com/questions/13821708/

(Tested only on Python 2.6.2, but according to the link above, Python 2.7.3 has the same problem.)
msg180146 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-01-17 18:32
What is in sample.txt? I cannot reproduce this with a source build
(Windows 7, 64-bit pgo build):

Python 2.7.3+ (default, Jan 17 2013, 20:26:24) [MSC v.1500 64 bit (AMD64)] on win32                          
Type "help", "copyright", "credits" or "license" for more information.                                       
>>> f = open('sample.txt')                                                                                   
>>> f.read(3)                                                                                                
'abc'                                                                                                        
>>> f.read(3)                                                                                                
'def'                                                                                                        
>>> f.tell()                                                                                                 
6L
msg180147 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-01-17 19:05
I am afraid I cannot reproduce with the 2.7.3 64-bit installer either:

C:\>python27\python.exe
Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open("sample.txt")
>>> f.read(3)
'xxx'
>>> f.read(3)
'xxx'
>>> f.tell()
6L
msg180301 - (view) Author: Catalin Iacob (catalin.iacob) * Date: 2013-01-20 16:45
Could it be that Raymond's file had Unix line endings?

I tried to reproduce this, picked a file on my disk and indeed I got a negative number, but that file has Unix line endings. This is documented at http://docs.python.org/2/library/stdtypes.html#file.tell so probably there's nothing to do then.

As for Armin's report in msg180145, even though it's not intuitive, this matches ftell's behavior on Windows, as documented in the Remarks section of http://msdn.microsoft.com/en-us/library/0ys3hc0b%28v=vs.100%29.aspx. The tell() method on fileobjects is explicitly documented as matching ftell behavior: "Return the file’s current position, like stdio‘s ftell()". So even though it's not intuitive at all, it's probably better to leave it as is. tell() returns the intuitive non zero position when opening with 'a' on Python3 and on Python 2.7 when using io.open so it's fixed for the future anyway.
msg186529 - (view) Author: Giacomo Alzetta (bakuriu) Date: 2013-04-10 20:11
I can't find any mention of this behaviour in python3's documentation, nor any reference to ftell(). Is it only well hidden or was it deleted by accident?
msg186551 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-04-11 09:29
Giacomo, which behaviour are you talking about?
FWIW, there is no such tell() issue in Python 3 and it doesn't use ftell().
msg186567 - (view) Author: Giacomo Alzetta (bakuriu) Date: 2013-04-11 16:42
I can reproduce a similar behaviour, but instead of negative values I obtain huge values(which may as well be a "negative" unsigned converted to a python int).

See this stackoverflow question: http://stackoverflow.com/questions/15934950/python-file-tell-giving-strange-numbers

If it doesn't use ftell() then this might mean that there is a new bug in the current implementation? I can reproduce the results of the questioner, on a windows7 machine with python3.3.0, with the following code:

    with open('C:/Users/Giacomo/test', 'wb') as f:
        f.write(b'hello\r\n\r\n-data1:blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah\r\n\r\n\r\n-data2:blah blah blah blah blah blah blah blah blah blah blah\r\n-data3: Empty\r\n\r\n-data4: Empty')
    with open('C:/Users/Giacomo/test', 'rt') as f:
        for line in iter(f.readline, ''):
            print(f.tell())

Which outputs:

    7
    9
    18446744073709551714
    99
    101
    164
    179
    181
    194

As I explained in my answer I thought the 1844... was due to the "ftell() bug", but if that's not it I'm at a loss. Could you explain this results with the current implementation?

By the way: it seems to be a python3.3 bug, since python3.2.3 is not affected.
msg186570 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2013-04-11 17:42
Giacomo, that's not related to the 2.7 problem. In Python 3, the result of f.tell() is not a file position, but a special value that tells the decoder to pick up when you seek().
msg186590 - (view) Author: Giacomo Alzetta (bakuriu) Date: 2013-04-11 20:40
The documentation for python 3.3.1 states, at http://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files, states:

    f.tell() returns an integer giving the file object’s current position in the file, **measured in bytes from the beginning of the file**.

And regarding files opened in text mode the only statement is:

    In text files (those opened without a b in the mode string), only seeks relative to the beginning of the file are allowed (the exception being seeking to the very file end with seek(0, 2)).

Even though in the io module it is documented that it may not be the number of bytes.

I think the former documentation should be changed, since it's simply wrong. Sorry for the disturb, but in the beginning I thought it was related.
msg186820 - (view) Author: Sijin Joseph (sijinjoseph) Date: 2013-04-13 20:15
Attaching a patch with a doc update to the tutorial, specifying that

1. The return value from f.tell is an opaque number for text files.
2. When seeking using text files the offset value needs to come from f.tell(). http://docs.python.org/3.4/library/io.html#io.TextIOBase.seek
msg193951 - (view) Author: Roundup Robot (python-dev) Date: 2013-07-30 19:53
New changeset 81bc2d64c006 by R David Murray in branch '3.3':
#16273: Fix tutorial discussion of seek/tell (opaque text-mode values).
http://hg.python.org/cpython/rev/81bc2d64c006

New changeset c7d9a2159c6c by R David Murray in branch 'default':
Merge: #16273: Fix tutorial discussion of seek/tell (opaque text-mode values).
http://hg.python.org/cpython/rev/c7d9a2159c6c
msg193952 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-07-30 19:56
Committed the doc patch.  Thanks Sijin.
History
Date User Action Args
2013-07-30 19:56:03r.david.murraysetstatus: open -> closed

nosy: + r.david.murray
messages: + msg193952

stage: patch review -> resolved
2013-07-30 19:53:45python-devsetnosy: + python-dev
messages: + msg193951
2013-04-13 21:14:08arigosetnosy: - arigo
2013-04-13 20:16:07pitrousetnosy: + docs@python
versions: + Python 3.3, Python 3.4, - Python 2.7
priority: high -> normal
assignee: docs@python
components: + Documentation, IO, - Interpreter Core
stage: test needed -> patch review
2013-04-13 20:15:20sijinjosephsetfiles: + 16273-doc-update.patch

nosy: + sijinjoseph
messages: + msg186820

keywords: + patch
2013-04-11 20:41:00brian.curtinsetnosy: - brian.curtin
2013-04-11 20:40:16bakuriusetmessages: + msg186590
2013-04-11 17:42:59benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg186570
2013-04-11 16:42:06bakuriusetmessages: + msg186567
2013-04-11 09:29:09pitrousetmessages: + msg186551
2013-04-10 20:11:31bakuriusetstatus: pending -> open
nosy: + bakuriu
messages: + msg186529

2013-03-28 10:34:52georg.brandlsetstatus: open -> pending
resolution: wont fix
2013-01-20 16:45:58catalin.iacobsetnosy: + catalin.iacob
messages: + msg180301
2013-01-20 05:23:54brian.curtinsetnosy: + brian.curtin
2013-01-17 19:05:04pitrousetmessages: + msg180147
2013-01-17 18:32:48skrahsetnosy: + skrah
messages: + msg180146
2013-01-17 17:39:32arigosetnosy: + arigo
messages: + msg180145
2013-01-07 17:44:43serhiy.storchakasetstage: test needed
2012-10-18 07:39:18serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg173236
2012-10-18 06:01:34pitrousetpriority: normal -> high
nosy: + pitrou
2012-10-18 02:08:42rhettingercreate