classification
Title: Windows - Popen (subprocess.py) does not call _handle.Close() at all
Type: behavior Stage: resolved
Components: Library (Lib), Windows Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Mateusz Klatt, eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2016-10-05 14:53 by Mateusz Klatt, last changed 2021-02-25 12:48 by eryksun. This issue is now closed.

Messages (2)
msg278129 - (view) Author: Mateusz Klatt (Mateusz Klatt) Date: 2016-10-05 14:53
_subprocess.TerminateProcess(self._handle, 1)

is not enough, on windows need to call self._handle.Close() after that

self._handle.Close() should be also called in __del__ - for the process es that ware not killed bu user, but terminated by themselves.

To reproduce... run popen in loop and observe file descriptors usage (SysInternals... handle -s -p <pid>)
msg278146 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-10-05 19:51
In 2.7, the _handle attribute is a _subprocess_handle object, which automatically calls CloseHandle when deallocated. For example:

    >>> p = subprocess.Popen('python -c "import time; time.sleep(120)"')

CreateProcess returns both the process handle and the thread handle. Python doesn't use the thread handle, so it explicitly closes it by calling ht.Close():

    Breakpoint 0 hit
    KERNELBASE!CloseHandle:
    00007ffb`a32fdf70 4883ec28        sub     rsp,28h
    0:000> kc 5
    Call Site
    KERNELBASE!CloseHandle
    python27!sp_handle_close
    python27!PyCFunction_Call
    python27!call_function
    python27!PyEval_EvalFrameEx
    0:000> g

(IMO, it should skip this step if creationflags contains CREATE_SUSPENDED. The thread handle makes it simpler to call ResumeThread.)

On the other hand, the process handle is deallocated implicitly when it's no longer referenced:

    >>> type(p._handle)
    <type '_subprocess_handle'>
    >>> hex(int(p._handle))
    '0x164L'

    >>> p.terminate()
    >>> del p

    Breakpoint 0 hit
    KERNELBASE!CloseHandle:
    00007ffb`a32fdf70 4883ec28        sub     rsp,28h
    0:000> kc 5
    Call Site
    KERNELBASE!CloseHandle
    python27!sp_handle_dealloc
    python27!dict_dealloc
    python27!subtype_dealloc
    python27!PyDict_DelItem
    0:000> r rcx
    rcx=0000000000000164

If the process handles aren't being closed in your case, then you're probably keeping a reference to the Popen instances.

P.S. A Windows handle is not a "file descriptor". Kernel handles are user-mode references to kernel objects. They aren't "files" unless the object happens to be a File object. A process handle references a kernel Process object.
History
Date User Action Args
2021-02-25 12:48:33eryksunsetstatus: open -> closed
resolution: out of date
stage: resolved
2016-10-05 19:51:06eryksunsetnosy: + eryksun
messages: + msg278146
2016-10-05 18:40:30SilentGhostsetnosy: + paul.moore, tim.golden, zach.ware, steve.dower
type: behavior
components: + Windows
2016-10-05 14:53:26Mateusz Klattcreate