This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients ezio.melotti, lemburg, methane, vstinner
Date 2022-03-22.13:30:06
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1647955806.51.0.282862246298.issue47000@roundup.psfhosted.org>
In-reply-to
Content
I propose:

* sys.getfilesystemencoding(): Python filesystem encoding, return "UTF-8" if the Python UTF-8 Mode is enabled

  * Implementation: PyConfig.filesystem_encoding
  * https://docs.python.org/dev/library/sys.html#sys.getfilesystemencoding
  * https://docs.python.org/dev/glossary.html#term-filesystem-encoding-and-error-handler
  * https://docs.python.org/dev/c-api/init_config.html#c.PyConfig.filesystem_encoding

* locale.getencoding(): Get the locale encoding, LC_CTYPE locale encoding or the Windows ANSI code page, *read at Python startup*. Ignore the Python UTF-8 Mode.

  * https://docs.python.org/dev/glossary.html#term-locale-encoding
  * Implementation: _Py_GetLocaleEncoding()
  * Existing *private* function: _locale._get_locale_encoding()

* locale.getencoding(current=True): Get the *current* locale encoding. The difference with locale.getencoding() is that on Unix, it gets the LC_CTYPE locale encoding at each call.

  * Implementation: _Py_GetLocaleEncoding() modified to ignore the UTF-8 mode.

None of these functions do locale.setlocale(locale.LC_CTYPE, "") to get the user preferred encoding.

Only the locale.getpreferredencoding() function uses locale.setlocale(locale.LC_CTYPE, "").

Usage of locale.getpreferredencoding() should be discouraged in the documentation, but I don't think that it can be deprecated and scheduled for removal right now: too much code rely on it :-(

---

So we have 3 encodings:

* Python filesystem encoding
* Locale encoding
* Current locale encoding

Examples of usage:

* Python filesystem encoding:

  * os.fsdecode() / os.fsencode()
  * C: PyUnicode_EncodeFSDefault() / PyUnicode_DecodeFSDefault()

* Locale encoding

  * _locale._get_locale_encoding()
  * On Unix, os.device_encoding()
  * To initialize PyConfig.stdio_encoding and PyConfig.filesystem_encoding

* Current locale encoding

  * PyUnicode_EncodeLocale() / PyUnicode_DecodeLocale()
  * "current_locale" parameter of private _Py_EncodeLocaleEx() / _Py_DecodeLocaleEx()
History
Date User Action Args
2022-03-22 13:30:06vstinnersetrecipients: + vstinner, lemburg, ezio.melotti, methane
2022-03-22 13:30:06vstinnersetmessageid: <1647955806.51.0.282862246298.issue47000@roundup.psfhosted.org>
2022-03-22 13:30:06vstinnerlinkissue47000 messages
2022-03-22 13:30:06vstinnercreate