classification
Title: install_egg_info fails with UnicodeEncodeError depending on locale
Type: behavior Stage: resolved
Components: Distutils, Distutils2 Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: duplicate
Dependencies: Superseder: distutils: set encoding to utf-8 for input and output files
View: 9561
Assigned To: tarek Nosy List: Arfrever, alexis, eric.araujo, hagen, mgorny, pje, serhiy.storchaka, tarek, vstinner
Priority: normal Keywords:

Created on 2011-05-16 12:21 by hagen, last changed 2018-07-11 07:32 by serhiy.storchaka. This issue is now closed.

Messages (4)
msg136084 - (view) Author: Hagen Fürstenau (hagen) Date: 2011-05-16 12:21
With issue 10419 fixed, I've run into the next distutils unicode bug: The command "install_egg_info" doesn't specify an encoding when opening the ".egg-info" file for writing. Depending on the locale, this may result in something like the following:

$ python setup.py install
[...]
Traceback (most recent call last):
  File "setup.py", line 67, in <module>
    main()
  File "setup.py", line 62, in main
    cmdclass={"test":TestCommand},
  File "/home/hagen/src/python/Lib/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/home/hagen/src/python/Lib/distutils/dist.py", line 919, in run_commands
    self.run_command(cmd)
  File "/home/hagen/src/python/Lib/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/home/hagen/src/python/Lib/distutils/command/install.py", line 583, in run
    self.run_command(cmd_name)
  File "/home/hagen/src/python/Lib/distutils/cmd.py", line 315, in run_command
    self.distribution.run_command(command)
  File "/home/hagen/src/python/Lib/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/home/hagen/src/python/Lib/distutils/command/install_egg_info.py", line 44, in run
    self.distribution.metadata.write_pkg_file(f)
  File "/home/hagen/src/python/Lib/distutils/dist.py", line 1033, in write_pkg_file
    file.write('Author: %s\n' % self.get_contact() )
UnicodeEncodeError: 'ascii' codec can't encode character '\xfc' in position 15: ordinal not in range(128)


I guess some encoding (UTF-8?) should be specified in the "run" method of "install_egg_info".
msg136085 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-05-16 12:26
install_egg_info.run() uses f = open(target, 'w') to create the .egg-info file and so the file uses the locale encoding. I don't know which module/program reads the file, so I don't know if the "locale encoding" is a good choice.
msg136110 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-16 16:10
The pkg_resources module in setuptools or distribute reads those files.

In 3.3, packaging.database reads them too (it’s an implementation of PEP 376 with b/w compat with egg-info).  It may work just fine in 3.x (it uses text I/O), but when we backport packaging to distutils2 for 2.4+, we’ll probably run into the problem again.
msg321412 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-11 07:32
Fixed in issue9561.
History
Date User Action Args
2018-07-11 07:32:32serhiy.storchakasetstatus: open -> closed

superseder: distutils: set encoding to utf-8 for input and output files
nosy: + serhiy.storchaka

messages: + msg321412
type: crash -> behavior
resolution: duplicate
stage: resolved
2011-06-22 13:34:23mgornysetnosy: + mgorny
2011-05-16 16:10:23eric.araujosetnosy: + alexis, pje
messages: + msg136110
components: + Distutils2
2011-05-16 15:49:53Arfreversetnosy: + Arfrever
2011-05-16 12:26:31vstinnersetnosy: + vstinner
messages: + msg136085
2011-05-16 12:21:35hagencreate