classification
Title: Deprecate os.kill() on Windows
Type: behavior Stage: needs patch
Components: Windows Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, giampaolo.rodola, jpe, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2015-04-14 15:24 by jpe, last changed 2016-02-12 14:16 by giampaolo.rodola.

Messages (9)
msg240905 - (view) Author: John Ehresman (jpe) * Date: 2015-04-14 15:24
os.kill() on Windows cannot act like it does on non-windows platforms because of differences in the underlying platforms.  I think only kill() with a signal number of 9 (terminate process unconditionally) and a signal number of 0 (test to see if process exists) could be implemented.  It isn't possibly to send the INT signal or a HUP signal to another process -- or at least it isn't possible without the use of questionable api's and code.

Windows specific functions would be added for terminateprocess() and generateconsolectrlevent().  os.kill() on non-windows platforms would be left alone.
msg241067 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-04-15 01:08
This feels like an unnecessary incompatibility between the platforms. I'd rather change the parameter values for CTRL+C events so we can distinguish when someone calls with that and then fix it internally on Windows.
msg241118 - (view) Author: John Ehresman (jpe) * Date: 2015-04-15 15:33
Part of the issue here is that GenerateConsoleCtrlEvent doesn't work like kill w/ SIGINT does on unix -- it only works if the the target process id is 0 and then it generates the signal in all processes that share the console.  An alternate proposal here is to expose the GenerateConsoleCtrlEvent functionality separately and only accept a signal number of 9 and maybe 0 in kill().
msg241134 - (view) Author: Eryk Sun (eryksun) * Date: 2015-04-15 17:41
> it only works if the the target process id is 0 and then it 
> generates the signal in all processes that share the console.

You can target a process group, which is a subset of the console's attached processes. That's why the console host, conhost.exe, routes dispatching the event through the session CSRSS.EXE, which handles bookkeeping of Windows processes and threads, including groups.

CSRSS creates a thread in each target, which starts at kernel32!CtrlRoutine. If a process is being debugged this routine raises a DBG_CONTROL_C or DBG_CONTROL_BREAK exception. Next it calls the registered handlers. The default handler, if called, exits the process with the exit code set to STATUS_CONTROL_C_EXIT (0xC000013A):

    0:000> u kernel32!DefaultHandler l3
    kernel32!DefaultHandler:
    00000000`76f31290 4883ec28        sub     rsp,28h
    00000000`76f31294 b93a0100c0      mov     ecx,0C000013Ah
    00000000`76f31299 ff15e1bb0500    call    qword ptr
                                              [kernel32!
                                               _imp_RtlExitUserProcess 
                                               (00000000`76f8ce80)]

Example:

    import os
    import sys
    import signal
    import subprocess
    import threading

    STATUS_CONTROL_C_EXIT = 0xC000013A

    p = subprocess.Popen([sys.executable, '-i'],
                         stdin=subprocess.PIPE,
                         stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE,
                         creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
    p.stderr.read(1) # poor man's WaitForInputIdle

    os.kill(p.pid, signal.CTRL_BREAK_EVENT)
    watchdog = threading.Timer(5, p.terminate)
    watchdog.start()
    exitcode = p.wait()
    watchdog.cancel()

    if exitcode < 0:
        exitcode += 2 ** 32
    assert exitcode == STATUS_CONTROL_C_EXIT
msg241135 - (view) Author: John Ehresman (jpe) * Date: 2015-04-15 17:53
GenerateConsoleCtrlEvent has different limitations for CTRL_BREAK_EVENT and CTRL_C_EVENT according to MSDN; I was referring to the CTRL_C_EVENT limitations.  Which python level signal handler will CTRL_BREAK_EVENT trigger?
msg241156 - (view) Author: Eryk Sun (eryksun) * Date: 2015-04-15 20:27
> Which python level signal handler will CTRL_BREAK_EVENT 
> trigger?

The CRT maps it to a non-standard signal, SIGBREAK (21). It's defined in the signal module.

> GenerateConsoleCtrlEvent has different limitations for 
> CTRL_BREAK_EVENT and CTRL_C_EVENT according to MSDN; 
> I was referring to the CTRL_C_EVENT limitations.  

When creating a process group, the ConsoleFlags value in the process parameters is set to ignore CTRL_C_EVENT. You'll still get a DBG_CONTROL_C exception if a debugger is attached, but otherwise kernel32!CtrlRoutine ignores Ctrl+C. As you say, the docs state that "[CTRL_C_EVENT] cannot be generated for process groups". But it *does* work if you remove the ignore flag. This flag gets set/unset by assigning/removing a NULL handler.

Here's a revised example to enable Ctrl+C in the child, disable Python's INT handler, and then generate CTRL_C_EVENT to kill the child and verify that the exit code is STATUS_CONTROL_C_EXIT.

    import os
    import sys
    import signal
    import subprocess
    import threading

    STATUS_CONTROL_C_EXIT = 0xC000013A

    p = subprocess.Popen([sys.executable, '-i', '-c',
                          'import ctypes, signal;'
                          'kernel32 = ctypes.windll.kernel32;'
                          'kernel32.SetConsoleCtrlHandler(None, 0);'
                          'signal.signal(signal.SIGINT, signal.SIG_DFL)'],
                         stdin=subprocess.PIPE,
                         stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE,
                         creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
    p.stderr.read(1) # poor man's WaitForInputIdle

    os.kill(p.pid, signal.CTRL_C_EVENT)
    watchdog = threading.Timer(5, p.terminate)
    watchdog.start()
    exitcode = p.wait()
    watchdog.cancel()

    if exitcode < 0:
        exitcode += 2 ** 32
    assert exitcode == STATUS_CONTROL_C_EXIT
msg241158 - (view) Author: John Ehresman (jpe) * Date: 2015-04-15 20:43
Interesting -- I didn't know about removing the ignore flag in the child process.  The question is whether this is close enough to the kill w/ SIGINT behavior on unix to use the same name.  I think that there are enough differences to warrant a Windows specific function that calls GenerateConsoleCtrlEvent.
msg256166 - (view) Author: Eryk Sun (eryksun) * Date: 2015-12-10 07:20
The signal module switched to using an enum for signal values: 

    >>> print(*map(repr, sorted(signal.Signals)), sep='\n')
    <Signals.CTRL_C_EVENT: 0>
    <Signals.CTRL_BREAK_EVENT: 1>
    <Signals.SIGINT: 2>
    <Signals.SIGILL: 4>
    <Signals.SIGFPE: 8>
    <Signals.SIGSEGV: 11>
    <Signals.SIGTERM: 15>
    <Signals.SIGBREAK: 21>
    <Signals.SIGABRT: 22>

We can use the console API when passed the CTRL_C_EVENT or CTRL_BREAK_EVENT enum. It may be easier to implement this version of os.kill in Python code instead of C. We can add the necessary WinAPI functions to the _winapi module.

BTW, I don't think it would be useful to implement the POSIX signal 0 test. Windows readily recycles process and thread IDs. These values are allocated out of the system's PspCidTable handle table. When a thread or process exits, the associated handle table entry is added to a freelist to be reused. So checking whether a process exists with a given PID is all but meaningless. Maybe the process you're interested in died and a new process was assigned the same ID.

To back that up, the following local kernel debugging session demonstrates that the kernel's PspCidTable is implemented as a handle table.

    Python 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37)
    [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os
    >>> os.getpid()
    3004
    >>> os.system('kd -kl')

    Microsoft (R) Windows Debugger Version 10.0.10240.9 AMD64
    Copyright (c) Microsoft Corporation. All rights reserved.

    [...]

    lkd> ;as pid 3004
    lkd> ;as cidtab ((nt!_HANDLE_TABLE *)@@(poi(nt!PspCidTable)))
    lkd> ;as tables ((void **)(${cidtab}->TableCode & ~0x11UI64))
    lkd> ;as tabnum ((${pid} / 4) / 256)
    lkd> ;as eindex ((${pid} / 4) % 256)
    lkd> ;as hentry ((nt!_HANDLE_TABLE_ENTRY *)${tables}[${tabnum}] + ${eindex})
    lkd> ;as ptrbit ((${hentry}->ObjectPointerBits << 4) | (0xffffUI64 << 48))
    lkd> ;as eprobj ((nt!_EPROCESS *)${ptrbit})

eprobj evaluates to the kernel process object that's assigned to the handle table entry based on the given PID. To complete the circle, check that the process object really does have this PID:

    lkd> ?? ${eprobj}->UniqueProcessId
    void * 0x00000000`00000bbc
    lkd> ?? 0xbbc
    int 0n3004

and that the image filename is python.exe:

    lkd> ?? (char *)${eprobj}->ImageFileName
    char * 0xffffe001`647564c8
     "python.exe"
msg260183 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2016-02-12 14:16
See also #26350.
History
Date User Action Args
2016-02-12 14:16:44giampaolo.rodolasetnosy: + giampaolo.rodola
messages: + msg260183
2015-12-10 07:20:10eryksunsetstage: needs patch
messages: + msg256166
versions: + Python 3.6, - Python 3.5
2015-04-15 20:43:57jpesetmessages: + msg241158
2015-04-15 20:27:26eryksunsetmessages: + msg241156
2015-04-15 17:53:56jpesetmessages: + msg241135
2015-04-15 17:41:43eryksunsetnosy: + eryksun
messages: + msg241134
2015-04-15 15:33:37jpesetmessages: + msg241118
2015-04-15 01:08:17steve.dowersetmessages: + msg241067
2015-04-14 15:24:21jpecreate