Title: test_memoryio fails for py3k on windows
Type: behavior Stage: patch review
Components: Tests, Windows Versions: Python 3.1
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: jaraco, kristjan.jonsson, ocean-city, pitrou
Priority: release blocker Keywords: patch

Created on 2009-04-01 11:53 by kristjan.jonsson, last changed 2009-06-05 02:29 by jaraco. This issue is now closed.

File name Uploaded Description Edit
memio.patch pitrou, 2009-04-04 12:36
Messages (8)
msg84999 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2009-04-01 11:53 test_memoryio fails various tests, including:
Traceback (most recent call last):
  File "D:\pydev\python\branches\py3k\lib\test\", line 
516, in test_issue5265
    self.assertEqual(, "a\nb\n")
AssertionError: 'a\n\nb\n' != 'a\nb\n'

Windows Vista.
msg85368 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-04-04 02:13
I'm sure, but with following test code,

def run(module):
    print("//", module)
    memio = module.StringIO(newline=None)
    # The C StringIO decodes newlines in write() calls, but the Python
    # implementation only does when reading.  This function forces them to
    # be decoded for testing.
    def force_decode():
        print("-------->", repr(memio.getvalue()))
        print("========>", repr(
    def print_newlines():
    print_newlines() # None
    print_newlines() # "\n"
    print_newlines() # ("\n", "\r\n")
    print_newlines() # ("\r", "\n", "\r\n")

def main():
    import _pyio, _io

if __name__ == '__main__':


I get result

// <module '_pyio' from 'e:\python-dev\py3k\l
--------> 'a\r\n'
========> 'a\n'
--------> 'a\r\nb\r\r\n'
========> 'a\nb\n\n'
('\r', '\r\n')
--------> 'a\r\nb\r\r\nc\rd'
========> 'a\nb\n\nc\nd'
('\r', '\r\n')
// <module 'io' (built-in)>
--------> 'a\n'
========> 'a\n'
--------> 'a\nb\n'
========> 'a\nb\n'
('\n', '\r\n')
--------> 'a\nb\nc\nd'
========> 'a\nb\nc\nd'
('\r', '\n', '\r\n')


Maybe universal new line decode behavior is inverse
between _pyio and _io?

That is, _pyio's write() converts '\n' to platform new line, and _io's
write() converts platform new line to '\n'.
msg85388 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-04-04 12:30
It looks like there is a disagreement between c-io and py-io as to what
the newline parameter in StringIO means.

I think the c-io version makes more sense (but I'm biased, since I wrote
it), while the py-io results are quite difficult to interpret.
msg85389 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-04-04 12:36
Can you test with the following patch?
msg85390 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-04-04 12:52
Buildbots should be green for the release.
msg85395 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-04-04 14:07
I confirmed the patch works.
msg85396 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-04-04 14:10
Committed in r71151, thanks!
msg88918 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2009-06-05 02:29
Patch is also applicable to #6127.
Date User Action Args
2009-06-05 02:29:22jaracosetnosy: + jaraco
messages: + msg88918
2009-04-04 14:10:02pitrousetstatus: open -> closed
resolution: fixed
messages: + msg85396
2009-04-04 14:07:37ocean-citysetmessages: + msg85395
2009-04-04 12:53:14pitrousettype: behavior
stage: patch review
2009-04-04 12:52:57pitrousetpriority: release blocker

messages: + msg85390
2009-04-04 12:36:56pitrousetfiles: + memio.patch
keywords: + patch
messages: + msg85389
2009-04-04 12:30:55pitrousetmessages: + msg85388
2009-04-04 02:23:17ocean-citysetnosy: + pitrou
2009-04-04 02:13:51ocean-citysetnosy: + ocean-city
messages: + msg85368
2009-04-01 11:53:29kristjan.jonssoncreate