Author vstinner
Recipients Sworddragon, larry, lemburg, loewis, ncoghlan, pitrou, r.david.murray, terry.reedy, vstinner
Date 2013-12-08.11:45:22
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAMpsgwbdGZMQnZMPrrx7F68HgZe+fCft4MH8xzJTeJckpRh13Q@mail.gmail.com>
In-reply-to <1386458541.9327.5.camel@fsol>
Content
2013/12/8 Antoine Pitrou <report@bugs.python.org>:
>> Python uses the fact that the filesystem encoding is the locale
>> encoding in various places.
>
> The patch doesn't change that.

You wrote: "-> With the patch: utf-8 utf-8 utf-8 ANSI_X3.4-1968", so
os.get sys.getfilesystemencoding() != locale.getpreferredencoding().
Or said differently, the filesystem encoding is different than the
locale encoding.

So please read again my following message which list real bugs:
https://mail.python.org/pipermail/python-dev/2010-October/104509.html

If you want to use a filesystem encoding different than the locale
encoding, you have to patch Python where Python assumes that the
filesystem encoding is the locale encoding, to fix all these bugs.
Starts with:

- PyUnicode_DecodeFSDefaultAndSize()
- PyUnicode_EncodeFSDefault()
- _Py_wchar2char()
- _Py_char2wchar()

It should be easier to change this function if the FS != locale only
occurs when FS is "UTF-8". On Mac OS X, Python always use UTF-8 for
the filesystem encoding, it doesn't care of the locale encoding. See
_Py_DecodeUTF8_surrogateescape() in unicodeobject.c, you may reuse it.

With a better patch, I can do more experiment to check if they are
other tricky bugs.

Does at least your patch pass the whole test suite with LANG=C?
History
Date User Action Args
2013-12-08 11:45:23vstinnersetrecipients: + vstinner, lemburg, loewis, terry.reedy, ncoghlan, pitrou, larry, r.david.murray, Sworddragon
2013-12-08 11:45:22vstinnerlinkissue19846 messages
2013-12-08 11:45:22vstinnercreate