This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: distutils doesn't correctly read UTF-8 content from config files
Type: behavior Stage: resolved
Components: Distutils, Library (Lib), Windows Versions: Python 3.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: delivrance, dstufft, eric.araujo, esuarezsantana, matrixise, paul.moore, steve.dower, tim.golden, vlcinsky, zach.ware
Priority: normal Keywords:

Created on 2017-12-05 16:46 by delivrance, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
setup.cfg delivrance, 2017-12-05 17:10
setup.py delivrance, 2017-12-05 17:10
Pull Requests
URL Status Linked Edit
PR 4727 closed delivrance, 2017-12-05 16:46
Messages (10)
msg307668 - (view) Author: Dan (delivrance) Date: 2017-12-05 16:46
On Windows, distutils doesn't correctly read UTF-8 content from config files (setup.cfg).

Seems like the issue is located on the line reading the files via the ConfigParser; simply adding 'encoding="UTF-8"' as argument fixes the problem for me: https://github.com/python/cpython/pull/4727

On Linux it seems to be working fine.
msg307669 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2017-12-05 16:57
Can you give an example setup.cfg file, setup.py command and the full error message?
msg307670 - (view) Author: Dan (delivrance) Date: 2017-12-05 17:13
I've attached the files.

Run using 'python setup.py sdist'.
The resulting PKG-INFO will contain incorrect data:

Summary: délivrance
Author: Dan Tès

The expected output is:

Summary: délivrance
Author: Dan Tès
msg307860 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2017-12-08 17:53
`metadata` in setup.cfg is not supported directly by distutils.  Can you provide a setup.py script that shows the problem without setuptools?
msg310674 - (view) Author: Jan Vlcinsky (vlcinsky) Date: 2018-01-25 13:43
The fix shall go to

https://github.com/python/cpython/blob/2812d3d99287c50bab99625d7240bcf1c2e32369/Lib/distutils/dist.py#L406

where `parser.read(filename)`

shall be changed to `parser.read(filename, encoding="utf-8")`

This assumes that the setup.cfg shall be UTF-8 encoded what I thing is correct assumption.

Alternative assumptions are (and I do not find them good):

- assume the file is encoded as current console is (this is not deterministic and is direct cause of this issue)
- let user to specify the encoding somewhere around (this requires extra step and does not bring any value)
msg338272 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2019-03-18 17:53
Repeat: `metadata` in setup.cfg is not supported directly by distutils.  Can you provide a setup.py script that shows the problem without setuptools?
msg341139 - (view) Author: Eduardo Suarez-Santana (esuarezsantana) * Date: 2019-04-30 09:04
In my opinion, 'setup.cfg' is setuptools stuff and not part of python project. Anyway I think it is a good idea to read it as UTF-8.

Setuptools seems to be defaulting to UTF-8: https://github.com/pypa/setuptools/pull/1735
msg341553 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2019-05-06 16:04
Hi,

I have closed the PR because there is no feedback from deliverance and the CLA must be signed for the processing. Once the CLA will be signed, the PR could be re-opened.

Thank you so much,
msg341605 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2019-05-06 19:00
No, setup.py and setup.cfg are invented by distutils and extended by setuptools.
msg386290 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-02-03 18:10
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools
History
Date User Action Args
2022-04-11 14:58:55adminsetgithub: 76404
2021-02-03 18:10:58steve.dowersetstatus: open -> closed
resolution: out of date
messages: + msg386290

stage: resolved
2019-05-06 19:00:12eric.araujosetmessages: + msg341605
2019-05-06 16:04:28matrixisesetnosy: + matrixise

messages: + msg341553
versions: - Python 3.6
2019-04-30 09:04:34esuarezsantanasetnosy: + esuarezsantana
messages: + msg341139
2019-03-18 17:53:12eric.araujosetmessages: + msg338272
2018-01-25 13:43:29vlcinskysetnosy: + vlcinsky
messages: + msg310674
2017-12-08 17:53:48eric.araujosetmessages: + msg307860
2017-12-05 17:13:50delivrancesetmessages: + msg307670
2017-12-05 17:10:58delivrancesetfiles: + setup.py
2017-12-05 17:10:52delivrancesetfiles: + setup.cfg
2017-12-05 16:57:45eric.araujosetversions: - Python 3.4, Python 3.5, Python 3.8
2017-12-05 16:57:32eric.araujosetmessages: + msg307669
2017-12-05 16:46:26delivrancecreate