I've noticed an issue (or user error) in which Python a call that otherwise usually works in the __del__ step of a class will freeze when the Python interpreter is exiting.

I've attached sample code that I've ran against Python 3.9.1 on Windows 10.

The code below runs a process and communicates via the pipe.

class SubprocTest(object):
	def run(self):
		print("run")
		proc_args = ["cmd.exe"]
		self._process = subprocess.Popen(proc_args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
	
	def __del__(self):
		print("__del__")
		if self._process is not None:
			self.terminate()
	
	def terminate(self):
		print("terminate")
		self._process.communicate(input=b"exit\n", timeout=1)
		print("kill")
		self._process.kill()
		self._process = None

if __name__ == "__main__":
	s = SubprocTest()
	s.run()
	del s
	print("s done")
	
	t = SubprocTest()
	t.run()
	print("t done")


Current output:
run
__del__
terminate
kill
s done
run
t done
__del__
terminate
<<<<<< hangs indefinitely here, even though timeout=1

Expected output:
run
__del__
terminate
kill
s done
run
t done
__del__
terminate
kill


In normal circumstances, when you del the object and force a run of __del__(), the process ends properly and the terminate() method completes.

When the Python interpreter exits, Python calls the __del__() method of the class.  In this case, the terminate() never completes and the script freezes indefinitely on the communicate()

It's not a subprocess bug, per se. It's due to creating the stdout/stderr worker threads from the __del__ finalizer while the interpreter is shutting down. Minimal reproducer, confirmed in both Linux and Windows:

    import threading

    class C:
        def __del__(self):
            t = threading.Thread(target=print, args=('spam',), daemon=True)
            t.start()

    c = C()
    #del c # uncomment to prevent hanging

eryksun, wow, that's speedy analysis, but there might be more to it.  I went and tested a bunch of test cases.  my subrocess code doesn't seem to hang on Linux where the thread example code does?

Linux - Python 3.6.8 - your threading example DOESN'T hang
Linux - Python 3.6.8 - My subprocess code also DOESN'T hang

Linux - Python 3.8.5 - thread example HANGs
Linux - Python 3.8.5 - My subprocess code also DOESN'T hang

Windows - Python 3.9.1 - thread example HANGs
Windows - Python 3.9.1 - subprocess code HANGs

> my subrocess code doesn't seem to hang on Linux where the 
> thread example code does?

Reader threads for stdout and stderr are only used in Windows, since there's no equivalent to select/poll for synchronous pipes in Windows. 

Without using threads, I/O with synchronous pipes requires a busy loop. It has to poll PeekNamedPipe() to get the number of bytes available to read without blocking. For stdin, on the other hand, the Windows API does not allow getting the WriteQuotaAvailable from the PipeLocalInformation [1]. Without knowing how much can be written without blocking, the input pipe would have to be made non-blocking, which in turn requires an inner loop that tries to write a successively smaller chunk size to stdin until either it succeeds or the size is reduced to 0. 

If we wanted to change communicate() in Windows to not use threads and not require a busy loop, we'd have to switch to using named pipes opened in asynchronous (overlapped) mode on our side of each pipe. But that's an integration problem. For normal use as proc.stdout, etc, we would need an adapter or a new raw file type that implements a synchronous interface by waiting for I/O completion and maintaining its own file pointer. Such files would return the Windows file handle for fileno(), like sockets do, since a C file descriptor requires a synchronous-mode file.

---

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_file_pipe_local_information