The locale encoding is not portable, packaging should use UTF-8 instead.

Attached patch is a draft to workaround "LANG=C ./python -m test test_packaging" failures. I'm not sure that my test in Metadata.write_file() of packaging.metadata is a good idea.

Moreover, packaging should also maybe use the surrogateescape error handler, but I propose to decide that later and in another issue. We may use this error handler only to read files (like the Python makefile).

For write a complete patch, *all* calls to open() in Lib/packaging/* (including Lib/packaging/tests/*) should be checked.

distutils uses the locale encoding, which is UTF-8 on Mac OS X and most Linux setup (but never on Windows). But distutils cannot be fixed, whereas packaging is a new module and can be fixed.

See also issue #9561 (distutils: set encoding to utf-8 for input and output files) and #6011 (python doesn't build if prefix contains non-ascii characters).

If you are curious, read also #8611 (Python3 doesn't support locale different than utf8 and an non-ASCII path (POSIX)) and #9425 (Rewrite import machinery to work with unicode paths).

Looks good, please commit this

I ported a distutils fix (for non-ASCII path) into packaging:

New changeset cc5cfeaa4a8d by Victor Stinner in branch 'default':
Issue #10419, issue #6011: port 6ad356525381 fix from distutils to packaging
http://hg.python.org/cpython/rev/cc5cfeaa4a8d

+        if not isinstance(fileobject, StringIO):
+            encoding = codecs.lookup(fileobject.encoding).name

IMO you should try to get the "encoding" attribute and silence the AttributeError instead.

(also, I'm not even sure why you're adding this check)

New changeset fe740c1cee02 by Victor Stinner in branch 'default':
Issue #12112: packaging reads and writes setup.cfg using UTF-8
http://hg.python.org/cpython/rev/fe740c1cee02

New changeset 01d61096140a by Victor Stinner in branch 'default':
Issue #12112: packaging reads/writes metadata using UTF-8
http://hg.python.org/cpython/rev/01d61096140a

What is the MainProgram.inspect_file() function in packaging.create? Which kind of file should it process? => What is the encoding of the input files?

That's not used anymore, I am going to strip it

New changeset a636cb1b7f84 by Victor Stinner in branch 'default':
Issue #12112: fix the encoding of setup.py in the packaging module
http://hg.python.org/cpython/rev/a636cb1b7f84

I can run LANG=C python -m test test_packaging successfully.  Can we close this?