classification
Title: distutils fail to install unicode-encoded files with POSIX locale
Type: behavior Stage: resolved
Components: Distutils Versions: Python 3.1, Python 3.2
process
Status: closed Resolution: duplicate
Dependencies: Superseder: distutils: set encoding to utf-8 for input and output files
View: 9561
Assigned To: tarek Nosy List: eric.araujo, mgorny, tarek
Priority: normal Keywords:

Created on 2010-10-08 11:49 by mgorny, last changed 2010-10-18 18:08 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
build.log mgorny, 2010-10-08 11:49 Complete Gentoo build log
Messages (2)
msg118188 - (view) Author: Michał Górny (mgorny) * Date: 2010-10-08 11:49
I was reported an issue where the user was unable to install my package [1] using python3.1. I was able to guess the issue comes from locale being set to POSIX and reproduce it.

It seems that the reasons are utf8-encoded characters in the installed script file. All the project files carry encoding declaration as per PEP-0263. They install fine with python2 but fail both with 3.1 and 3.2.

I think the build system should be locale-agnostic, and work correctly even if LC_ALL=C. If it's necessary to use text mode for scripts, I think a similar encoding guessing method should be used as in python itself.

I'm attaching a complete build log (for multiple versions of Python). Below I'm repasting the backtrace:

Traceback (most recent call last):
  File "setup.py", line 35, in <module>
    'Topic :: System :: Installation/Setup'
  File "/usr/lib64/python3.1/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/usr/lib64/python3.1/distutils/dist.py", line 919, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python3.1/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.1/distutils/command/build.py", line 128, in run
    self.run_command(cmd_name)
  File "/usr/lib64/python3.1/distutils/cmd.py", line 315, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python3.1/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.1/distutils/command/build_scripts.py", line 51, in run
    self.copy_scripts()
  File "/usr/lib64/python3.1/distutils/command/build_scripts.py", line 83, in copy_scripts
    first_line = f.readline()
  File "/usr/lib64/python3.1/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 127: ordinal not in range(128)

[1] http://github.com/mgorny/smart-live-rebuild
msg119049 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-18 18:08
Thanks for the report.  This is a duplicate, you can add yourself to the nosy list on the superseder bug to track status.
History
Date User Action Args
2010-10-18 18:08:42eric.araujosetstatus: open -> closed
resolution: duplicate
messages: + msg119049

superseder: distutils: set encoding to utf-8 for input and output files
stage: resolved
2010-10-08 11:49:37mgornycreate