Author eryksun
Recipients BreamoreBoy, damiro, eryksun, pitrou, steve.dower, tim.golden, tim.peters, vstinner, zach.ware
Date 2015-04-13.11:46:43
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1428925604.62.0.395365306698.issue19050@psf.upfronthosting.co.za>
In-reply-to
Content
The default (and standards-violating) behavior of the Windows CRT is to kill the process for a bad file descriptor, instead of just setting errno to EBADF. To work around this PyOS_StdioReadline needs to to check _Py_Verify_fd before calling fflush or writing to stderr. A similar check was added to my_fgets in 3.2.5 (see issue 14433), but it wasn't backported to Python 2.

The REPL in 3.x still uses C FILE streams, so this problem absolutely affects 3.x. (Except 3.5.0a2 and 3.5.0a3 have a hack that hides the problem. The hack has since been removed, so the problem has returned out of hiding in recent builds.) Previously I omitted the stack trace for brevity because it's virtually identical to 2.7, but here it is as justification for adding 3.x back to the issue's versions field.

    C:\Program Files\Python34>cdb -xi ld python

    Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
    Copyright (c) Microsoft Corporation. All rights reserved.

    CommandLine: python
    Symbol search path is: symsrv*symsrv.dll*C:\Symbols*http://msdl.microsoft.com/download/symbols
    Executable search path is:
    (1184.11f0): Break instruction exception - code 80000003 (first chance)
    ntdll!LdrpDoDebuggerBreak+0x30:
    00000000`76e8cb70 cc              int     3
    0:000> bp ntdll!NtTerminateProcess
    0:000> g

    Python 3.4.2 (v3.4.2:ab2c023a9432, Oct  6 2014, 22:16:31) [MSC v.1600 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os
    >>> os.close(2)

    Breakpoint 0 hit
    ntdll!NtTerminateProcess:
    00000000`76e31570 4c8bd1          mov     r10,rcx
    0:000> kc
    Call Site
    ntdll!NtTerminateProcess
    KERNELBASE!TerminateProcess
    MSVCR100!invalid_parameter
    MSVCR100!invalid_parameter_noinfo
    MSVCR100!write
    MSVCR100!flush
    MSVCR100!fflush_nolock
    MSVCR100!fflush
    python34!PyOS_StdioReadline
    python34!PyOS_Readline
    python34!tok_nextc
    python34!tok_get
    python34!PyTokenizer_Get
    python34!parsetok
    python34!PyParser_ParseFileObject
    python34!PyParser_ASTFromFileObject
    python34!PyRun_InteractiveOneObject
    python34!PyRun_InteractiveLoopFlags
    python34!PyRun_AnyFileExFlags
    python34!run_file
    python34!Py_Main
    python!__tmainCRTStartup
    kernel32!BaseThreadInitThunk
    ntdll!RtlUserThreadStart


Additionally, here's a trace that demonstrates the silent handler hack that's present in 3.5.0a2:

    C:\Program Files\Python35>cdb -xi ld python

    Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
    Copyright (c) Microsoft Corporation. All rights reserved.

    CommandLine: python
    Symbol search path is: symsrv*symsrv.dll*C:\Symbols*http://msdl.microsoft.com/download/symbols
    Executable search path is:
    (1040.113c): Break instruction exception - code 80000003 (first chance)
    ntdll!LdrpDoDebuggerBreak+0x30:
    00000000`76e8cb70 cc              int     3
    0:000> bp ucrtbase!set_thread_local_invalid_parameter_handler
    0:000> bp python35!_silent_invalid_parameter_handler
    0:000> g

With this hack in place, new_threadstate unconditionally sets the invalid parameter handler to _silent_invalid_parameter_handler (again, this has since been removed):

    Breakpoint 0 hit
    ucrtbase!set_thread_local_invalid_parameter_handler:
    000007fe`e47c8740 4053            push    rbx
    0:000> kc
    Call Site
    ucrtbase!set_thread_local_invalid_parameter_handler
    python35!new_threadstate
    python35!_Py_InitializeEx_Private
    python35!Py_Main
    python!__scrt_common_main_seh
    kernel32!BaseThreadInitThunk
    ntdll!RtlUserThreadStart
    0:000> g

    Python 3.5.0a2 (v3.5.0a2:0337bd7ebcb6, Mar  8 2015, 07:17:31) [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os
    >>> os.close(2)

    Breakpoint 1 hit
    python35!_silent_invalid_parameter_handler:
    00000000`5f6e9d60 c20000          ret     0
    0:000> kc
    Call Site
    python35!_silent_invalid_parameter_handler
    ucrtbase!invalid_parameter
    ucrtbase!write
    ucrtbase!_acrt_stdio_flush_nolock
    ucrtbase!fflush_nolock
    ucrtbase!fflush
    python35!PyOS_StdioReadline
    python35!PyOS_Readline
    python35!tok_nextc
    python35!tok_get
    python35!parsetok
    python35!PyParser_ASTFromFileObject
    python35!PyRun_InteractiveOneObject
    python35!PyRun_InteractiveLoopFlags
    python35!PyRun_AnyFileExFlags
    python35!run_file
    python35!Py_Main
    python!__scrt_common_main_seh
    kernel32!BaseThreadInitThunk
    ntdll!RtlUserThreadStart

I think for 3.5 the affected code needs to be bracketed by _Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH. This works on my personal build. See issue 23524 for more details regarding the new macros.
History
Date User Action Args
2015-04-13 11:46:44eryksunsetrecipients: + eryksun, tim.peters, pitrou, vstinner, tim.golden, damiro, BreamoreBoy, zach.ware, steve.dower
2015-04-13 11:46:44eryksunsetmessageid: <1428925604.62.0.395365306698.issue19050@psf.upfronthosting.co.za>
2015-04-13 11:46:44eryksunlinkissue19050 messages
2015-04-13 11:46:43eryksuncreate