This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients ronaldoussoren, vstinner
Date 2012-11-05.22:18:21
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1352153903.84.0.995057035576.issue16416@psf.upfronthosting.co.za>
In-reply-to
Content
Since the changeset 45079ad1e260 (issue #4388), command line arguments are decoded from UTF-8 instead of the locale encoding. Functions of Python/fileutils.c are still using the locale encoding.

It does not work: see issue #16218. On Mac OS X, in the command line "python script.py", the filename "script.py" is decoded from UTF-8 (by _Py_DecodeUTF8_surrogateescape) but then it is passed to _Py_fopen() which encodes the filename to the locale encoding (ex: ISO-8859-1 if $LANG, $LC_CTYPE and $LC_ALL environment variables are not set). The result is mojibake and Python fails to open the script.

Attached patch modifies function of Python/fileutils.c to use UTF-8 to encode and decode filenames, instead of the locale encoding on Mac OS X.

I don't know yet if Module/getpath.c should also decode paths from UTF-8 instead of the locale encoding on Mac OS X. We may expose _Py_decode_filename().
History
Date User Action Args
2012-11-05 22:18:23vstinnersetrecipients: + vstinner, ronaldoussoren
2012-11-05 22:18:23vstinnersetmessageid: <1352153903.84.0.995057035576.issue16416@psf.upfronthosting.co.za>
2012-11-05 22:18:23vstinnerlinkissue16416 messages
2012-11-05 22:18:22vstinnercreate