This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: os.device_encoding(fd) should support any console fd in Windows
Type: behavior Stage:
Components: Extension Modules, Interpreter Core, IO, Windows Versions: Python 3.10, Python 3.9, Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2021-03-06 15:04 by eryksun, last changed 2022-04-11 14:59 by admin.

Messages (1)
msg388201 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-03-06 15:04
In Windows, os.device_encoding() is hard coded to map file descriptor 0 and descriptors 1 and 2 respectively to the console's input and output code page if isatty(fd) is true. But isatty() is true for any character device, such as "NUL". Also any fd might be a console, by way of dup(), dup2() -- or open() with the device names "CON", "CONIN$", and "CONOUT$".

The correct device encoding of a console file is needed for use with and os.write(). It's also necessary for io.TextIOWrapper() if PYTHONLEGACYWINDOWSSTDIO is set.

_Py_device_encoding() in Python/fileutils.c should use _get_osfhandle() to get the OS handle, and, if it's a character-device file, determine the code page to return, if any, depending on whether it's an input or output console file. For example:

PyObject *
_Py_device_encoding(int fd)
#if defined(MS_WINDOWS)
    HANDLE handle;
    DWORD temp;
    UINT cp = 0;

        handle = (HANDLE)_get_osfhandle(fd);
    if (handle == INVALID_HANDLE_VALUE ||
            GetFileType(handle) != FILE_TYPE_CHAR)

    /* GetConsoleMode requires a console handle. */
    if (!GetConsoleMode(handle, &temp)) {
        /* Assume access denied implies output. */
        if (GetLastError() == ERROR_ACCESS_DENIED)
            cp = GetConsoleOutputCP();
    } else {
        if (GetNumberOfConsoleInputEvents(handle, &temp)) {
            cp = GetConsoleCP();
        } else {
            cp = GetConsoleOutputCP();

    if (cp == CP_UTF8) {
        return PyUnicode_FromString("UTF-8");
    } else if (cp != 0) {
        return PyUnicode_FromFormat("cp%u", (unsigned int)cp);
    } else {
    if (isatty(fd)) {
        return _Py_GetLocaleEncodingObject();
    } else {
#endif  /* defined(MS_WINDOWS) */
Date User Action Args
2022-04-11 14:59:42adminsetgithub: 87587
2021-03-06 15:04:20eryksuncreate