Python cannot access msvcrt's putwch() when using manage.py syncdb

To reproduce:
1. Call manage.py syncdb and try to create a new superuser
2. It crashes after inputting email (or before asking for the password)

Reproducible with 3.5a3, seems to be a regression from 3.4.3. Downgrading fixed the issue

Edit: I am using a windows 8.1 system and django 1.8

Can you reproduce this without involving Django?  That would make it more likely that someone will have time to take a look at it.

The new CRT used by 3.5 has a separate header, corecrt_wconio.h, for declarations shared by conio.h and wchar.h. Thus the _WCONIO_DEFINED macro is no longer defined, and consequently PC/msvcrtmodule.c skips defining getwch, getwche, putwch, and ungetwch.

I guess this check was added to support DOS-based Windows 9x. NT-based Windows would always support wide-character console I/O.

You're right, we should be able to remove the ifdef for these (or hide them behind MS_WINDOWS if necessary).

Had the same problem but was able to fix this by rewriting the win_getpass method in getpass.py:

def win_getpass(prompt='Password: ', stream=None):
    """Prompt for password with echo off, using Windows getch()."""
    if sys.stdin is not sys.__stdin__:
        return fallback_getpass(prompt, stream)
    import msvcrt
    for c in prompt:
        msvcrt.putch(c.encode('utf-8'))
    pw = ""
    while 1:
        c = msvcrt.getch()
        if c == '\r'.encode('utf-8') or c == '\n'.encode('utf-8'):
            break
        if c == '\003'.encode('utf-8'):
            raise KeyboardInterrupt
        if c == '\b'.encode('utf-8'):
            pw = pw[:-1]
        else:
            pw = pw + c.decode('utf-8')
    msvcrt.putch('\r'.encode('utf-8'))
    msvcrt.putch('\n'.encode('utf-8'))
    return pw

I'm setting this to release blocker because it sounds like a simple fix and I don't think we should release with these basic windows functions missing.  If you (Steve) don't think it is important for the beta you could set it to deferred blocker.

I'll just remove the ifdefs. We don't support any Windows versions that don't have these functions.

Short of hard-coding a list of expected functions and using hasattr, anyone have any ideas about how to test stuff like this? I kind of feel like alpha/beta releases are the most efficient way to find these.

New changeset d56a941865fb by Steve Dower in branch 'default':
Issue #23995: Removes _WCONIO_DEFINED check as the wchar_t console functions are always available.
https://hg.python.org/cpython/rev/d56a941865fb

Shouldn't tests for these functions be part of our testing of the io module or similar?

I wouldn't have thought so, since the IO stack is entirely portable, at least from the Python side of things. This would have to have been a test that somehow knows about optional functions and notifies you if they're missing but without failing the run. I don't think that's feasible, at least in this case, since there's no way to know that the function should be there.

Potentially we could set up a test to ensure that we don't remove public functions by dumping a complete list, but that's beyond the scope of this one issue.

You'll note that the problem shows up in the getpass module, which does have tests, but which does not have a test that discovers this.  That's because writing tests that *use* these functions is not really practical :)

Someone could open an issue about testing for the functions that are supposed to be available.  There could also be another issue for gettpass to test for the availability of the functions it actually uses.

Steve, looks like you forgot to close the issue?  Or were you waiting for agreement about the tests?

Was waiting for agreement or opposition, but I intended to close it within 24 hours if nothing was raised :)

Testing getpass shouldn't be that difficult if you use ctypes to call WriteConsoleInput [1]. For example:

    from ctypes import *
    from ctypes.wintypes import *

    kernel32 = WinDLL('kernel32')

    IN, OUT, INOUT = 1, 2, 3
    KEY_EVENT = 0x0001
    STD_INPUT_HANDLE  = -10
    STD_OUTPUT_HANDLE = -11
    STD_ERROR_HANDLE  = -12
    INVALID_HANDLE_VALUE = HANDLE(-1).value

    class KEY_EVENT_RECORD(Structure):
        class UCHAR(Union):
            _fields_ = (('UnicodeChar', WCHAR),
                        ('AsciiChar',   CHAR))
        _fields_ = (('bKeyDown',          BOOL),
                    ('wRepeatCount',      WORD),
                    ('wVirtualKeyCode',   WORD),
                    ('wVirtualScanCode',  WORD),
                    ('uChar',             UCHAR),
                    ('dwControlKeyState', DWORD))

    class INPUT_RECORD(Structure):
        class EVENT(Union):
            _fields_ = (('KeyEvent', KEY_EVENT_RECORD),)
        _fields_ = (('EventType', WORD),
                    ('Event',     EVENT))

    PINPUT_RECORD = POINTER(INPUT_RECORD)

    class HANDLE_IHV(HANDLE):
        @classmethod
        def _check_retval_(cls, retval):
            if retval.value == INVALID_HANDLE_VALUE:
                raise WinError(get_last_error())
            return retval.value

    def errcheck_bool(result, func, args):
        if not result:
            raise WinError(get_last_error())
        return args

    def WINAPI(name, dll, restype, *argspec):
        if argspec:
            argtypes = tuple(p[0] for p in argspec)
            paramflags = tuple(p[1:] for p in argspec)
        else:
            argtypes = paramflags = ()
        prototype = WINFUNCTYPE(restype, *argtypes, use_last_error=True)
        func = prototype((name, dll), paramflags)
        if restype in (BOOL, HANDLE):
            func.errcheck = errcheck_bool
        setattr(dll, name, func)

    WINAPI('GetStdHandle', kernel32, HANDLE_IHV,
            (DWORD, IN, 'nStdHandle'))

    WINAPI('WriteConsoleInputW', kernel32, BOOL,
           (HANDLE,         IN,     'hConsoleInput'),
           (PINPUT_RECORD,  IN,     'lpBuffer'),
           (DWORD,          IN,     'nLength'),
           (LPDWORD,        OUT,    'lpNumberOfEventsWritten'))

    def write_console_input(s):
        hInput = kernel32.GetStdHandle(STD_INPUT_HANDLE)
        recs = (INPUT_RECORD * len(s))()
        for c, rec in zip(s, recs):
            rec.EventType = KEY_EVENT
            rec.Event.KeyEvent.bKeyDown = True
            rec.Event.KeyEvent.wRepeatCount = 1
            rec.Event.KeyEvent.uChar.UnicodeChar = c
        return kernel32.WriteConsoleInputW(hInput, recs, len(recs))

    if __name__ == '__main__':
        import getpass
        test_input = 'Console test input.\n'
        n = write_console_input(test_input)
        assert n == len(test_input)
        read_input = getpass.getpass()
        assert read_input == test_input.rstrip()

This requires that python.exe is run with an attached console (conhost.exe), i.e. the process can't be created as a DETACHED_PROCESS [2] nor can it be run using pythonw.exe without first calling AllocConsole.
    
[1]: https://msdn.microsoft.com/en-us/library/ms687403
[2]: https://msdn.microsoft.com/en-us/library/ms684863