classification
Title: 'python -u' yields trailing carriage return '\r' (Python2 for Windows)
Type: behavior Stage: resolved
Components: Interpreter Core, Windows Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: msp, r.david.murray, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2014-07-09 12:51 by msp, last changed 2020-04-26 06:14 by zach.ware. This issue is now closed.

Files
File name Uploaded Description Edit
input.py msp, 2014-07-09 12:51 sample file to reproduce error
binary.patch vstinner, 2014-07-28 21:49 review
Messages (7)
msg222613 - (view) Author: Matthias St.Pierre (msp) Date: 2014-07-09 12:51
raw_input yields trailing carriage return ('\r') when C:\Python27\python.exe is called using the -u option

How to reproduce the error: save the attached file 'input.py' which contains the following lines

-- begin input.py --

i = raw_input("enter y or n: ")
print(repr(i))
print([ord(c) for c in i])

-- end input.py --

then run the two following commands from a windows command shell
	python input.py
	python -u input.py
and compare the output. (see transcript 1 below)

The same bug affects also the interactive mode: start 'python -u' and enter an arbitrary command. You will get a syntax error at the end of line. (see transcript 2 below)


-- begin transcript 1 of cmd session --

C:\Temp>where python
C:\Python27\python.exe

C:\Temp>python --version
Python 2.7.8

C:\Temp>type input.py

i = raw_input("enter y or n: ")
print(repr(i))
print([ord(c) for c in i])

C:\Temp>python input.py
enter y or n: y
'y'
[121]
-- end transcript 1 of cmd session --


-- begin transcript 2 of cmd session --

C:\Temp>python -u input.py
enter y or n: y
'y\r'
[121, 13]

C:\Temp>python -u
Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print "hello world"
  File "<stdin>", line 1
    print "hello world"
                       ^
SyntaxError: invalid syntax
>>>

-- end transcript 2 of cmd session --
msg222614 - (view) Author: Matthias St.Pierre (msp) Date: 2014-07-09 12:56
correction: the markers for transcripts 1 and 2 were inserted at the wrong location; here's the correction:


-- begin transcript 1 of cmd session --

C:\Temp>where python
C:\Python27\python.exe

C:\Temp>python --version
Python 2.7.8

C:\Temp>type input.py

i = raw_input("enter y or n: ")
print(repr(i))
print([ord(c) for c in i])

C:\Temp>python input.py
enter y or n: y
'y'
[121]

C:\Temp>python -u input.py
enter y or n: y
'y\r'
[121, 13]


-- end transcript 1 of cmd session --


-- begin transcript 2 of cmd session --

C:\Temp>python -u
Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print "hello world"
  File "<stdin>", line 1
    print "hello world"
                       ^
SyntaxError: invalid syntax
>>>

-- end transcript 2 of cmd session --
msg223339 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-07-17 14:35
This is the documented behavior of the -u option.  It puts the streams in binary mode "on systems where it matters", which would be windows.  That is, universal newline processing is disabled when you use -u.

Note that this is no longer an issue in python3: there, -u only affects buffering, and not the binary/text mode (because in python3 there is always a distinction between binary and text mode, regardless of platform).
msg223347 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-07-17 16:24
The bug is not on print, but raw_input().

In Python 3, I worked on the following issues to support fully binary standard streams:

 - #10841: "binary stdio"
 - #11272: "input() has trailing carriage return on windows", fixed in Python 3.2.1
 - #11395: "print(s) fails on Windows with long strings", fixed in Python 3.2.1
 - #13119: "Newline for print() is \n on Windows, and not \r\n as expected", will be fixed in Python 3.2.4 and 3.3 (not released yet)

It looks like this issue is the same than #11272: input() removes '\n' but not '\r'.
msg224194 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-07-28 21:49
I backported and adapted most Python 3 fixes related to the Windows binary mode for stdout/stderr used when Python is started with the -u command line option. See attached binary.patch.

My change to file.write() looks wrong because the function returns None and does not loop until all bytes are written.

By the way, string_write() doesn't check if fwrite(n) wrote less than n bytes or failed with an error.

See also the issue #21090 which improves error handling in the file object.
msg224197 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-07-28 22:09
While testing binary.patch, test_subprocess failed because of the issue #19612. I backported the fix:

changeset:   91905:039ac3f01c4e
branch:      2.7
parent:      91895:bffa0b8a16e8
user:        Victor Stinner <victor.stinner@gmail.com>
date:        Tue Jul 29 00:04:54 2014 +0200
files:       Lib/subprocess.py Misc/NEWS
description:
Issue #19612: subprocess.communicate() now also ignores EINVAL when using at
least two pipes.
msg367300 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2020-04-26 06:14
It's not entirely clear from a quick read whether this was actually fixed or not (probably?), but as 2.7 is out of support it no longer matters :)
History
Date User Action Args
2020-04-26 06:14:17zach.waresetstatus: open -> closed

nosy: + zach.ware
messages: + msg367300

resolution: out of date
2014-07-28 22:09:42vstinnersetmessages: + msg224197
2014-07-28 21:49:41vstinnersetfiles: + binary.patch
keywords: + patch
messages: + msg224194
2014-07-17 16:24:14vstinnersetstatus: closed -> open
resolution: not a bug -> (no value)
messages: + msg223347
2014-07-17 14:35:50r.david.murraysetstatus: open -> closed

nosy: + r.david.murray
messages: + msg223339

resolution: not a bug
stage: resolved
2014-07-12 17:36:33ezio.melottisetnosy: + vstinner
2014-07-09 12:56:28mspsetmessages: + msg222614
2014-07-09 12:51:58mspcreate