classification
Title: py3k: io.StringIO.getvalue() returns \r\n
Type: behavior Stage:
Components: Library (Lib), Windows Versions: Python 3.0
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, amaury.forgeotdarc, gvanrossum
Priority: normal Keywords: patch

Created on 2007-08-26 22:04 by amaury.forgeotdarc, last changed 2007-09-02 20:05 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
stringio.diff amaury.forgeotdarc, 2007-08-26 22:04
no_write_nl_translate.patch alexandre.vassalotti, 2007-08-27 22:38
Messages (6)
msg55314 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2007-08-26 22:04
io.StrinIO.getvalue() correctly decodes bytes from the underlying
buffer, but does not translate \r\n to \n.

Python 3.0x (py3k, Aug 26 2007, 14:39:16) [MSC v.1400 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import io
>>> a = io.StringIO()
>>> a.write('\n')
2
>>> a.getvalue()
'\r\n'

The attached patch corrects this and adds a test.
msg55323 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2007-08-26 23:26
As far as I know, StringIO should not do any string transformations.

From PEP-3116 "New I/O", last paragraph of the section "Text I/O":
> Another implementation, StringIO, creates a file-like TextIO
> implementation without an underlying Buffered I/O object. [...]  It
> does not support encodings or newline translations; you always read
> back exactly the characters you wrote.
msg55324 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2007-08-26 23:35
> As far as I know, StringIO should not do any string transformations.

(Not sure if you agree with the patch or not)
That's why the current behaviour is not correct: When I write('\n'),
getvalue() currently returns '\r\n'.
msg55341 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2007-08-27 22:38
> That's why the current behaviour is not correct: When I write('\n'),
> getvalue() currently returns '\r\n'.

Oh, I missed your example in your initial message. So yes, I agree that
StringIO shouldn't translate newlines like that. I attached a patch that
should fix the bug.

However, I would favor removing the "newline" keyword argument, instead.
msg55359 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2007-08-28 09:00
Here is a new version of the class, which removes the 'newline' argument
from the constructor.
I also removed the 'encoding' argument, since this is really an
implementation detail of the underlying buffer.

Index: Lib/io.py
===================================================================
--- Lib/io.py	(revision 57564)
+++ Lib/io.py	(working copy)
@@ -1390,10 +1390,10 @@

     # XXX This is really slow, but fully functional

-    def __init__(self, initial_value="", encoding="utf-8", newline=None):
+    def __init__(self, initial_value=""):
         super(StringIO, self).__init__(BytesIO(),
-                                       encoding=encoding,
-                                       newline=newline)
+                                       encoding="utf-8",
+                                       newline='\n')
         if initial_value:
             if not isinstance(initial_value, basestring):
                 initial_value = str(initial_value)
msg55428 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-08-29 18:32
I'ev fixed this slightly differently, by simply changing the *default*
of the newline argument to StringIO to be "\n".  This is a good default;
but I see no reason to prevent users from messing with it if they have a
need.
History
Date User Action Args
2007-09-02 20:05:22loewissetkeywords: + patch
2007-08-29 18:32:14gvanrossumsetstatus: open -> closed
resolution: fixed
messages: + msg55428
nosy: + gvanrossum
2007-08-28 09:00:55amaury.forgeotdarcsetmessages: + msg55359
2007-08-27 22:38:50alexandre.vassalottisetfiles: + no_write_nl_translate.patch
messages: + msg55341
2007-08-26 23:35:10amaury.forgeotdarcsetmessages: + msg55324
2007-08-26 23:26:30alexandre.vassalottisetnosy: + alexandre.vassalotti
messages: + msg55323
2007-08-26 22:04:03amaury.forgeotdarccreate