This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients ZackerySpytz, eryksun, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Date 2019-06-26.15:15:50
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1561562151.11.0.833046423852.issue37413@roundup.psfhosted.org>
In-reply-to
Content
sys._enablelegacywindowsfsencoding() was added late in PEP 529 design "just in case" if something goes wrong. But I'm not aware of anyone using it. Do we want to keep supporting the *legacy* Windows filesystem encoding (ANSI code page) forever? IMHO using UTF-8 is a way more practical solution to design portable applications working unmodified on Windows *and* Unix. Well, it's the purpose of the PEP 529.

I propose to deprecate sys._enablelegacywindowsfsencoding() and PYTHONLEGACYWINDOWSFSENCODING environment variable in Python 3.9 and remove it from Python 3.10. Calling sys._enablelegacywindowsfsencoding() would emit a DeprecationWarning in 3.9.

I dislike sys._enablelegacywindowsfsencoding() because it can lead to mojibake: filenames decoded from the ANSI code page but then encoded to UTF-8. In the PEP 587 "Python Initialization Configuration" I tried to ensure that encodings are set early: in a new "pre-initialization" phase. Encodings should not change after the pre-initialization.

--

By the way, I'm not aware of any issue with io._WindowsConsoleIO. Should we also deprecated PYTHONLEGACYWINDOWSSTDIO environment variable which opt-out from the new io._WindowsConsoleIO?

Extract of open() code in Modules/_io/_iomodule.c:

    /* Create the Raw file stream */
    {
        PyObject *RawIO_class = (PyObject *)&PyFileIO_Type;
#ifdef MS_WINDOWS
        PyConfig *config = &_PyInterpreterState_GET_UNSAFE()->config;
        if (!config->legacy_windows_stdio && _PyIO_get_console_type(path_or_fd) != '\0') {
            RawIO_class = (PyObject *)&PyWindowsConsoleIO_Type;
            encoding = "utf-8";
        }
#endif
        raw = PyObject_CallFunction(RawIO_class,
                                    "OsiO", path_or_fd, rawmode, closefd, opener);
    }
History
Date User Action Args
2019-06-26 15:15:51vstinnersetrecipients: + vstinner, paul.moore, tim.golden, zach.ware, eryksun, steve.dower, ZackerySpytz
2019-06-26 15:15:51vstinnersetmessageid: <1561562151.11.0.833046423852.issue37413@roundup.psfhosted.org>
2019-06-26 15:15:51vstinnerlinkissue37413 messages
2019-06-26 15:15:50vstinnercreate