Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subprocess with redirection fails after FreeConsole #69678

Closed
GeorgePrekas mannequin opened this issue Oct 27, 2015 · 3 comments
Closed

subprocess with redirection fails after FreeConsole #69678

GeorgePrekas mannequin opened this issue Oct 27, 2015 · 3 comments
Labels
OS-windows stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@GeorgePrekas
Copy link
Mannequin

GeorgePrekas mannequin commented Oct 27, 2015

BPO 25492
Nosy @pfmoore, @tjguk, @zware, @eryksun, @zooba
Superseder
  • bpo-3905: subprocess failing in GUI applications on Windows
  • Files
  • testcase.py: test case
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2015-12-05.19:43:17.258>
    created_at = <Date 2015-10-27.15:33:14.963>
    labels = ['type-bug', 'library', 'OS-windows']
    title = 'subprocess with redirection fails after FreeConsole'
    updated_at = <Date 2015-12-05.19:43:17.257>
    user = 'https://bugs.python.org/GeorgePrekas'

    bugs.python.org fields:

    activity = <Date 2015-12-05.19:43:17.257>
    actor = 'eryksun'
    assignee = 'none'
    closed = True
    closed_date = <Date 2015-12-05.19:43:17.258>
    closer = 'eryksun'
    components = ['Library (Lib)', 'Windows']
    creation = <Date 2015-10-27.15:33:14.963>
    creator = 'George Prekas'
    dependencies = []
    files = ['40868']
    hgrepos = []
    issue_num = 25492
    keywords = []
    message_count = 3.0
    messages = ['253547', '253573', '253574']
    nosy_count = 6.0
    nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'George Prekas']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '3905'
    type = 'behavior'
    url = 'https://bugs.python.org/issue25492'
    versions = ['Python 2.7', 'Python 3.4', 'Python 3.5', 'Python 3.6']

    @GeorgePrekas
    Copy link
    Mannequin Author

    GeorgePrekas mannequin commented Oct 27, 2015

    Under Windows, if I do FreeConsole and then subprocess.call with redirected stdin or stdout or stderr, then subprocess.call fails. The only solution is either DO NOT redirect any handle OR to redirect ALL of them. The problem lies in function _get_handles at subprocess.py:810 (Python 2.7.10), which does:

    • GetStdHandle
    • DuplicateHandle via _make_inheritable.

    DuplicateHandle fails if called on stdin, stdout or stderr after FreeConsole.

    $ cygstart /mnt/c/utils/Python35/python.exe testcase.py fails
    $ cygstart /mnt/c/utils/Python35/python.exe testcase.py works1
    $ cygstart /mnt/c/utils/Python35/python.exe testcase.py works2
    $ cygstart /mnt/c/utils/Python35/python.exe testcase.py works3

    @GeorgePrekas GeorgePrekas mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Oct 27, 2015
    @eryksun
    Copy link
    Contributor

    eryksun commented Oct 28, 2015

    For me, fails() lives up to its name in Windows 7, but it doesn't fail in Windows 10. It shouldn't fail in Windows 8, either.

    In Windows 8+ the console interface is implemented using a kernel device. Console handles reference virtual files on the ConDrv device, such as Connect, Input, and Output. The example doesn't fail because the I/O handles (Input and Output) are valid even if the main console handle (Connect) has been closed by calling FreeConsole.

    OTOH, prior to Windows 8, console I/O handles don't reference kernel objects. You may have noticed that the old API tags the values by setting the lower 2 bits (e.g. 3, 7, 11). This enables redirecting actions on console I/O handles to functions in the console host process (i.e. conhost.exe in Win 7, and csrss.exe in XP/Vista). For example, for a regular kernel object, the DuplicateHandle API is serviced by the system call NtDuplicateObject. But for a console handle, Windows instead sends the request to the console LPC port, to be dispatched to SrvDuplicateHandle in the console. Obviously this can't work after the client has called FreeConsole.

    Prior to Windows 8, this invalid-handle error can also be the result of running pythonw.exe from a console application such as python.exe or cmd.exe. See bpo-3905. In this case Windows is copying the values of the parent's standard handles into the pythonw.exe process parameters, but they're meaningless values since pythonw.exe doesn't attach to a console automatically. (You could of course manually call AllocConsole or AttachConsole.) The new implementation in Windows 8 and 10 is smart enough to initialize the standard handles to 0 in this case. Thus in Windows 10 Terry's example in msg220218 doesn't raise and exception, and p.stdout.read() == b'32\r\n'.

    A workaround for Python 3.4+ on older Windows versions would be to check os.get_handle_inheritable, which calls WinAPI GetHandleInformation 1:

        ERROR_INVALID_HANDLE = 0x0006
    
        if stdin is None:
            p2cread = _winapi.GetStdHandle(_winapi.STD_INPUT_HANDLE)
            try:
                os.get_handle_inheritable(p2cread)
            except OSError as e:
                if e.winerror != ERROR_INVALID_HANDLE:
                    raise
                p2cread = None
            if p2cread is None:
                p2cread, _ = _winapi.CreatePipe(None, 0)
                p2cread = Handle(p2cread)
                _winapi.CloseHandle(_)

    For 2.7, GetHandleInformation could be added to _subprocess. But then it may as well be added to Python 3's _winapi as well.

    @eryksun
    Copy link
    Contributor

    eryksun commented Oct 28, 2015

    I forgot to first check whether p2cread is None:

        ERROR_INVALID_HANDLE = 0x0006
        
        if stdin is None:
            p2cread = _winapi.GetStdHandle(_winapi.STD_INPUT_HANDLE)
            if p2cread is not None:
                try:
                    os.get_handle_inheritable(p2cread)
                except OSError as e:
                    if e.winerror != ERROR_INVALID_HANDLE:
                        raise
                    p2cread = None
            if p2cread is None:
                p2cread, _ = _winapi.CreatePipe(None, 0)
                p2cread = Handle(p2cread)
                _winapi.CloseHandle(_)

    @eryksun eryksun closed this as completed Dec 5, 2015
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    OS-windows stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants