classification
Title: io.StringIO newline param has wrong default
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: benjamin.peterson, couplewavylines, docs@python, hynek, pitrou, python-dev, serhiy.storchaka, stutzbach
Priority: normal Keywords: easy

Created on 2014-01-28 19:24 by couplewavylines, last changed 2014-02-02 21:52 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
no_translation.py couplewavylines, 2014-01-28 19:24 Demonstrates default StringIO newline translation behavior
no_translation_output.txt couplewavylines, 2014-01-28 19:26 Output of no_translation.py
Messages (7)
msg209577 - (view) Author: (couplewavylines) Date: 2014-01-28 19:24
In io.StringIO, the newline argument's default is currently documented as "newline=None" in the section header. However, it's described this way:  "The default is to do no newline translation."  The behavior of io.StringIO is consistent with this description (NO newline translation). 
The header should actually read "newline=''".
"newline=None" would mean there IS newline translation by default, which is not the case. Code sample attached as no_translation.py.
msg209579 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-28 19:37
No, actual default is '\n'.

>>> io.StringIO('abc\r\ndef\nghi\rklm').readlines()
['abc\r\n', 'def\n', 'ghi\rklm']
>>> io.StringIO('abc\r\ndef\nghi\rklm', newline=None).readlines()
['abc\n', 'def\n', 'ghi\n', 'klm']
>>> io.StringIO('abc\r\ndef\nghi\rklm', newline='').readlines()
['abc\r\n', 'def\n', 'ghi\r', 'klm']
>>> io.StringIO('abc\r\ndef\nghi\rklm', newline='\n').readlines()
['abc\r\n', 'def\n', 'ghi\rklm']
>>> io.StringIO('abc\r\ndef\nghi\rklm', newline='\r').readlines()
['abc\r', '\r', 'def\r', 'ghi\r', 'klm']
>>> io.StringIO('abc\r\ndef\nghi\rklm', newline='\r\n').readlines()
['abc\r\r\n', 'def\r\n', 'ghi\rklm']
msg209580 - (view) Author: (couplewavylines) Date: 2014-01-28 19:44
Serhiy, you're right, I now see the default behavior is "newline='\n'".

So, the header is still wrong, and the text is "The default is to do no newline translation" may be incorrect too?  Or just unclear (to me anyway)?
msg209581 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-28 19:51
On other hand, may be the behavior of io.StringIO is wrong. Because it is different from io.TextIOWrapper.

>>> io.TextIOWrapper(io.BytesIO(b'abc\r\ndef\nghi\rklm')).readlines()
['abc\n', 'def\n', 'ghi\n', 'klm']
>>> io.TextIOWrapper(io.BytesIO(b'abc\r\ndef\nghi\rklm'), newline=None).readlines()
['abc\n', 'def\n', 'ghi\n', 'klm']
>>> io.TextIOWrapper(io.BytesIO(b'abc\r\ndef\nghi\rklm'), newline='').readlines()
['abc\r\n', 'def\n', 'ghi\r', 'klm']
>>> io.TextIOWrapper(io.BytesIO(b'abc\r\ndef\nghi\rklm'), newline='\n').readlines()
['abc\r\n', 'def\n', 'ghi\rklm']
>>> io.TextIOWrapper(io.BytesIO(b'abc\r\ndef\nghi\rklm'), newline='\r').readlines()
['abc\r', '\ndef\nghi\r', 'klm']
>>> io.TextIOWrapper(io.BytesIO(b'abc\r\ndef\nghi\rklm'), newline='\r\n').readlines()
['abc\r\n', 'def\nghi\rklm']
msg209590 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-01-28 21:42
Using '\n' as default is normal: StringIOs are not stored on disk so there's no point in converting newlines by default (it's only slower while not improving interoperability). '\n' is the default line separator in Python.

So we should just fix the docs here.

The bug when using '\r' or '\r\n' should probably be fixed though (that would be a separate issue).
msg210021 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-02-02 21:51
New changeset 82cfab2ad98d by Antoine Pitrou in branch '3.3':
Issue #20423: fix documentation of io.StringIO's newline parameter
http://hg.python.org/cpython/rev/82cfab2ad98d

New changeset 69a2cc048c80 by Antoine Pitrou in branch '2.7':
Issue #20423: fix documentation of io.StringIO's newline parameter
http://hg.python.org/cpython/rev/69a2cc048c80

New changeset df2efd48227e by Antoine Pitrou in branch 'default':
Issue #20423: fix documentation of io.StringIO's newline parameter
http://hg.python.org/cpython/rev/df2efd48227e
msg210022 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-02 21:52
The docs are now fixed, thank you!
History
Date User Action Args
2014-02-02 21:52:34pitrousetstatus: open -> closed
resolution: fixed
messages: + msg210022

stage: needs patch -> resolved
2014-02-02 21:51:52python-devsetnosy: + python-dev
messages: + msg210021
2014-01-28 21:42:10pitrousetmessages: + msg209590
2014-01-28 19:51:04serhiy.storchakasetnosy: + pitrou, benjamin.peterson, stutzbach, hynek
messages: + msg209581
2014-01-28 19:44:20couplewavylinessetmessages: + msg209580
2014-01-28 19:37:23serhiy.storchakasetversions: + Python 2.7
nosy: + serhiy.storchaka

messages: + msg209579

keywords: + easy
stage: needs patch
2014-01-28 19:26:20couplewavylinessetfiles: + no_translation_output.txt
2014-01-28 19:24:38couplewavylinescreate