This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: buffered io seek() buggy
Type: behavior Stage: resolved
Components: IO Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, pakal, pitrou
Priority: normal Keywords: patch

Created on 2010-01-05 19:38 by pakal, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
Py26_relative_seek.patch pakal, 2010-02-01 19:58
Messages (11)
msg97275 - (view) Author: Pascal Chambon (pakal) * Date: 2010-01-05 19:38
I've noticed a severe bug in my python 2.6.2 io module, and looking at _buffered_raw_seek in http://svn.python.org/view/python/trunk/Modules/_io/bufferedio.c?view=markup, it seems the bug is still there in the C reimplementation of buffered io classes.

The problem is, if we seek a buffered stream with whence=os.SEEK_CUR, that call is directly transmitted to the underlying raw stream, which has not the same file pointer position as the buffered stream. So in the end, the file pointer doesn't get moved at the right absolute offset, but at an offset which depends of the state of the read head buffer or the write buffer.
When using os.SEEK_CUR, it would be necessary to call tell() or to manually compute offsets, so that the seek() works as expected.

Regards, 
Pascal
msg97276 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-01-05 19:46
Could you construct a test case?
(it's even better if you fix the offending code as well of course)
msg97287 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-01-06 01:41
The problem seems to be fixed in trunk (the future 2.7).
BufferedRandom.seek() contains a block labelled "# Undo read ahead." which does not exists in 2.6.

I reproduce the problem with a file opened with mode "rb+"

from io import open
open("tmpfile", "wb").write("abcdefgh" * 1024 + "X")
f = open("tmpfile", "rb+")
print f.read(1) # "a"
f.seek(0, 1)
print f.read(1) # should be "b", 2.6 returns "X"
msg97302 - (view) Author: Pascal Chambon (pakal) * Date: 2010-01-06 10:47
My bad, I had looked at _buffered_raw_seek, not buffered_seek >_<
Indeed, the code is OK in both trunk _io an _pyio modules.
And the SEEK_CUR flag (value : 1) seems more than sufficiently tested in test_io actually, for example in the function write_ops() :
http://svn.python.org/view/python/trunk/Lib/test/test_io.py?view=markup

So concerning python2.6, isn't that just possible to backport _pyio (and its test suite) to it (renamed as io.py) ? They seem to offer the same functionality, except that _pyio is much more robust than io.py (the code around seek(), in particular, is much more complete).
Just tell me, else I'll gather a patch for the seek() problem.
msg97303 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-01-06 10:53
> So concerning python2.6, isn't that just possible to backport _pyio
> (and its test suite) to it (renamed as io.py) ?

I would not be against it, but someone has to provide a patch.
The current _pyio.py might rely on other new behaviour of 2.7, so
perhaps it won't be that easy.
msg97305 - (view) Author: Pascal Chambon (pakal) * Date: 2010-01-06 11:26
Hum, with a selective merge (tortoiseSVN makes it easy), backporting _pyio should be doable in a decent time. Triaging pertinent tests should be more brain damaging :p
I'm looking at this.
msg97307 - (view) Author: Pascal Chambon (pakal) * Date: 2010-01-06 14:23
Well, here is a patch for the seek() methods of io module, in python2.6 maintenance branch.
Finally, I've only backported some assertions and the offset stuffs - I'm not comfortable enough about recent io refactorings to do more (it changed pretty quickly while I looked away, actually :p).
Tests have been patched too (bufferedrandom wasn't tested as other buffer classes), and to have the whole suite pass, I had to modify testWriteNonBlocking() as well (mock objects returned wrong values - this test is marked as unreliable in sources anyway).
If more is needed to patch the py2.6 branch, just tell me B-)
msg98681 - (view) Author: Pascal Chambon (pakal) * Date: 2010-02-01 19:57
BufferedRandom's seek() is still borken in the latest svn revision of python2.6 (I haven't checked python2.7 yet), so here is a new, smaller patch, tested with the 6 IO-related test suites this time, on win32 and linux.
msg98682 - (view) Author: Pascal Chambon (pakal) * Date: 2010-02-01 19:58
The patch itself...
msg105796 - (view) Author: Pascal Chambon (pakal) * Date: 2010-05-15 07:53
Hello

I advocate the inclusion of this patch to the 2.6 maintenance branch, because currently the io module in this branch (which is still the most recent 2.X version released) is simply broken. 
People using it will certainly encounter MAJOR file corruptions due to the buggy seek().

Or should we rather backport the much more evoluated io of trunk to py2.6 ?
msg105822 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-05-15 20:33
Committed in r81203. Thank you, Pascal!
History
Date User Action Args
2022-04-11 14:56:56adminsetgithub: 51889
2010-05-15 20:33:38pitrousetstatus: open -> closed
versions: - Python 3.1, Python 2.7, Python 3.2
messages: + msg105822

resolution: fixed
stage: resolved
2010-05-15 07:53:05pakalsetmessages: + msg105796
2010-02-01 19:58:08pakalsetfiles: + Py26_relative_seek.patch

messages: + msg98682
2010-02-01 19:57:21pakalsetmessages: + msg98681
2010-02-01 19:54:52pakalsetfiles: - seek_cur_buffers_py26.patch
2010-01-06 14:23:16pakalsetfiles: + seek_cur_buffers_py26.patch
keywords: + patch
messages: + msg97307
2010-01-06 11:26:04pakalsetmessages: + msg97305
2010-01-06 10:53:34pitrousetmessages: + msg97303
2010-01-06 10:47:46pakalsetmessages: + msg97302
2010-01-06 01:41:01amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg97287
2010-01-05 19:46:59pitrousetnosy: + pitrou
messages: + msg97276
2010-01-05 19:38:09pakalcreate