Title: encoding error for ' --author' when read via subprocess pipe
Type: behavior Stage: resolved
Components: Unicode Versions: Python 2.7
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: eric.araujo, lemburg, srid, tarek
Priority: normal Keywords:

Created on 2009-06-01 18:32 by srid, last changed 2010-11-25 19:28 by eric.araujo. This issue is now closed.

Messages (5)
msg88664 - (view) Author: Sridhar Ratnakumar (srid) Date: 2009-06-01 18:32
I ran 'python --author' for the pypi package "ll-orasql-0.6"
(whose author name has non-ascii characters) under subprocess.Popen and
this is what I get:

Traceback (most recent call last):
  File "/home/sridharr/as/pypm/bin/python", line 39, in <module>
  File "", line 50, in <module>
    package_dir={"ll": ""}
  File "/opt/ActivePython-2.6/lib/python2.6/distutils/", line
138, in setup
    ok = dist.parse_command_line()
  File "/opt/ActivePython-2.6/lib/python2.6/distutils/", line
456, in parse_command_line
    if self.handle_display_options(option_order):
  File "/opt/ActivePython-2.6/lib/python2.6/distutils/", line
704, in handle_display_options
    print value
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in
position 8: ordinal not in range(128)

When ran under the console, this exception is not seen. Otherwise, the
return code is 1 and nothing (apart from the above exception) is printed.

How to reproduce?

Download `ll-orasql-0.6` from PyPI and run the following in the Python

>>> subprocess.Popen('python --author', stdout=subprocess.PIPE,
msg88674 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2009-06-01 20:41
This is not a bug in distutils, but how print works when executed
through subprocess. 

here's a demo:

Create a file called "" with:

  # -*- coding: utf8 -*-
  print u'éééé'

Now another one called "" with:

  import subprocess

  subprocess.Popen('python', stdout=subprocess.PIPE,

Now launch test2:

  $ python 
  Traceback (most recent call last):
  File "", line 2, in <module>
    print u'éééé'
  UnicodeEncodeError: 'ascii' codec can't encode characters in position 
0-3: ordinal not in range(128)

I don't know about the internals of print, and I am not sure this is a
bug, so I'll put Marc-André in the loop.
msg121410 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-18 01:52
Sridhar, can you test with the envvar PYTHONIOENCODING set to UTF-8?
msg122171 - (view) Author: Sridhar Ratnakumar (srid) Date: 2010-11-22 22:30
"PYTHONIOENCODING=UTF-8 python" does work.
msg122398 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-25 19:28
I agree with Tarek this is not a bug, and there is a workaround, so I’m closing this.
Date User Action Args
2010-11-25 19:28:36eric.araujosetstatus: open -> closed
resolution: works for me
messages: + msg122398

stage: resolved
2010-11-22 22:30:26sridsetmessages: + msg122171
2010-11-18 01:52:50eric.araujosetnosy: + eric.araujo

messages: + msg121410
versions: + Python 2.7, - Python 2.6
2009-06-01 20:41:48tareksetnosy: + lemburg
messages: + msg88674

assignee: tarek ->
components: + Unicode, - Distutils
2009-06-01 18:32:26sridcreate