classification
Title: stdout.encoding not set when redirecting windows command line output
Type: behavior Stage:
Components: Windows Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Redoute, loewis
Priority: normal Keywords:

Created on 2012-03-04 09:42 by Redoute, last changed 2012-03-04 14:54 by loewis. This issue is now closed.

Messages (4)
msg154885 - (view) Author: Redoute (Redoute) Date: 2012-03-04 09:42
When running a python script from windows command line (cmd.exe) and redirecting its output, stdout.encoding is set to None and printing non-ascii chars fails. Encoding should be the same as without redirecting.
Example:

[Code unictest.py]
# -*- coding: utf-8 -*-

from sys import stdout, stderr
print >> stderr, 'stdout.encoding: ', stdout.encoding
print u'äöüß'
[/Code]

[windows command prompt]
C:\Daten>testunic.py
stdout.encoding:  cp850
äöüß

C:\Daten>testunic.py > testunic.txt
stdout.encoding:  None
Traceback (most recent call last):
  File "C:\Daten\Cmd\testunic.py", line 5, in <module>
    print u'├ñ├Â├╝├ƒ'
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: ordin
al not in range(128)
[/windows command prompt]
msg154888 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-03-04 12:05
That is impossible to implement. When outputting to a terminal window, the terminal's encoding is queried and used. When the Python script is run on its own, there may not even exist a terminal encoding.

In any case, this issue is resolved in Python 3 (not by setting the encoding to the terminal's encoding, but to the system code page). For 2.7, changing this would be a new feature, so I'm closing this as resolved - please upgrade to Python 3.
msg154892 - (view) Author: Redoute (Redoute) Date: 2012-03-04 13:32
Martin, thanks for your response.

I have to take it, although I am not really convinced. The system codepage probably differs from the console codepage (here 1152 vs. 850). Print statements on the other hand are typical for console programs. Calling python.exe (vs. pythonw.exe) means console program means print output has to use console codepage as default.

It is disappointing that output changes and programs fail because of a command line redirection.
msg154898 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-03-04 14:54
> I have to take it, although I am not really convinced. The system  
> codepage probably differs from the console codepage (here 1152 vs.  
> 850). Print statements on the other hand are typical for console  
> programs. Calling python.exe (vs. pythonw.exe) means console program  
> means print output has to use console codepage as default.

I think in practice, this statement is incorrect for most applications.
When they use print, they want to create text files, and those will
be opened with notepad (or some other text editor on Windows), and will
have to be in cp1252. So guessing that it should be cp850 will be incorrect
most of the time.

In any case, you have perfectly described why the initial design decision
in Python 2 was to refuse to guess.
History
Date User Action Args
2012-03-04 14:54:44loewissetmessages: + msg154898
2012-03-04 13:32:50Redoutesetmessages: + msg154892
2012-03-04 12:05:48loewissetstatus: open -> closed

nosy: + loewis
messages: + msg154888

resolution: fixed
2012-03-04 09:42:29Redoutecreate