This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eryksun
Recipients eryksun, malin, paul.moore, steve.dower, tim.golden, zach.ware, Александр Виноградов
Date 2016-04-29.14:11:53
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1461939115.45.0.0225333678599.issue26882@psf.upfronthosting.co.za>
In-reply-to
Content
python.dmp is from Windows 8.1 (NT 6.3):

    0:000> ?? @$peb->OSMajorVersion; ?? @$peb->OSMinorVersion
    unsigned long 6
    unsigned long 3

with the following command line and working directory:

    0:000> ?? @$peb->ProcessParameters->CommandLine
    struct _UNICODE_STRING
     ""C:\Python35\python.exe" -c print('hello')"
       +0x000 Length           : 0x54
       +0x002 MaximumLength    : 0x56
       +0x004 Buffer           : 0x01100a9e  ""C:\Python35\python.exe" -c print('hello')"

    0:000> ?? @$peb->ProcessParameters->CurrentDirectory
    struct _CURDIR
       +0x000 DosPath          : _UNICODE_STRING "C:\TestPlatform\integrationTests\"
       +0x008 Handle           : 0x00000024 Void

The STARTUPINFO dwFlags is set to STARTF_USESTDHANDLES (0x100), so the standard handles are from the STARTUPINFO hStdInput, hStdOutput, and hStdError:

    0:000> ?? @$peb->ProcessParameters->WindowFlags
    unsigned long 0x100

Standard input appears to be a console handle (starting in Windows 8 console handles are kernel handles, so we can inspect them in the debugger):

    0:000> ?? @$peb->ProcessParameters->StandardInput
    void * 0x00000018
    0:000> !handle 18 3
    Handle 00000018
      Type          File
      Attributes    0
      GrantedAccess 0x12019f:
             ReadControl,Synch
             Read/List,Write/Add,Append/SubDir/CreatePipe,ReadEA,WriteEA,ReadAttr,WriteAttr
      HandleCount   4
      PointerCount  131064

It looks like a console handle because it's open with both read and write access, which is required for the console API (e.g. ReadConsoleInput and WriteConsoleInput). 

Standard output appears to be a pipe or file, opened with only write access:

    0:000> ?? @$peb->ProcessParameters->StandardOutput
    void * 0x000011d4
    0:000> !handle 11d4 3
    Handle 000011d4
      Type          File
      Attributes    0
      GrantedAccess 0x120196:
             ReadControl,Synch
             Write/Add,Append/SubDir/CreatePipe,WriteEA,ReadAttr,WriteAttr
      HandleCount   2
      PointerCount  65536

The process is blocked on the NtQueryInformationFile [1] system call in the process of creating Python's sys.std* file objects.

    0:000> k
    ChildEBP RetAddr
    0109f354 74afa9a4 ntdll!NtQueryInformationFile+0xc
    0109f3ac 73e11b15 KERNELBASE!SetFilePointerEx+0x9c
    ...
    0109f450 73fe051e ucrtbase!_lseeki64+0x19
    ...
    0109f48c 740421e3 python35!_io_FileIO_tell+0x2d
    ...
    0109f508 73fe4948 python35!_buffered_raw_tell+0x20
    ...
    0109f53c 74053dc0 python35!_io_BufferedReader___init__+0x3f
    ...
    0109f648 740420ff python35!_io_open+0x90
    ...
    0109f710 740c37b4 python35!create_stdio+0xac
    0109f748 740c2871 python35!initstdio+0x264
    0109f760 73faf203 python35!_Py_InitializeEx_Private+0x351
    ...
    0109f808 1c7c11df python35!Py_Main+0x713

Specifically it's querying StandardInput (handle 0x18) for FilePositionInformation (0xe), which is stored in an 8-byte FILE_POSITION_INFORMATION struct (i.e. the size of a LARGE_INTEGER for the current position):

    0:000> dd @esp l6
    0109f358  74afa9a4 00000018 0109f37c 0109f388
    0109f368  00000008 0000000e

This is querying the value of the file pointer to initialize sys.stdin.buffer. 

Microsoft has been busy rolling out major updates to the console subsystem over the past few years. It could be a bug in the condrv.sys console device driver, which is new in Windows 8. It's hard to say what's going on without attaching a kernel debugger to inspect the problem.

Switching to a device driver required reimplementing the console API -- e.g. everything that used to get routed to LPC interprocess calls to conhost.exe now goes through I/O system calls. Specifically, console handles prior to Windows 8 were flagged by setting the lower 2 bits (e.g. 3, 7, 11, etc) to route calls to LPC-based functions where possible (e.g. ReadFile -> ReadConsoleA) or immediately fail. Most if not all of that specialized code is gone in Windows 8+. WinAPI calls with console handles now take the regular path to call NT I/O system calls such as NtReadFile, NtDeviceIoControlFile, and NtQueryInformationFile.

For example, in Windows 7 calling SetFilePointerEx to get or set the non-existent file pointer of a console handle fails immediately with STATUS_INVALID_HANDLE. This can be ignored harmlessly. But in the new console, we can do something silly like this (tested in Windows 10):

    >>> sys.stdin.seek(50)
    50
    >>> sys.stdin.tell()
    50

    >>> kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
    >>> h = kernel32.GetStdHandle(-10)
    >>> pos = ctypes.c_longlong()
    >>> kernel32.SetFilePointerEx(h, 0, ctypes.byref(pos), 1)
    1
    >>> pos.value
    50

(On a tangent, the new implementation also breaks Ctrl+C handling. It no longer sets ERROR_OPERATION_ABORTED when interrupting a ReadFile console read. This breaks Python's REPL and input function.)

[1]: https://msdn.microsoft.com/en-us/library/ff567052
History
Date User Action Args
2016-04-29 14:11:55eryksunsetrecipients: + eryksun, paul.moore, tim.golden, zach.ware, steve.dower, malin, Александр Виноградов
2016-04-29 14:11:55eryksunsetmessageid: <1461939115.45.0.0225333678599.issue26882@psf.upfronthosting.co.za>
2016-04-29 14:11:55eryksunlinkissue26882 messages
2016-04-29 14:11:53eryksuncreate