This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multiprocessing.Process file descriptor resource leak
Type: resource usage Stage:
Components: Documentation, Library (Lib) Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Robert Pierce, docs@python
Priority: normal Keywords:

Created on 2020-02-03 09:21 by Robert Pierce, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
proc_test.py Robert Pierce, 2020-02-03 09:21 test script to demonstrate resource leak
Messages (2)
msg361273 - (view) Author: Robert Pierce (Robert Pierce) Date: 2020-02-03 09:21
multiprocessing.Process opens a FIFO to the child. This FIFO is not documented the the Process class API and it's purpose is not clear from the documentation. It is a minor documentation bug that the class creates non-transparent resource utilization.

The primary behavioral bug is that incorrect handling of this FIFO creates a resource leak, since the file descriptor is not closed on join(), or even when the parent Process object goes out of scope.

The effect of this bug is that programs generating large numbers of Process objects will hit system resource limits of open file descriptors.
msg361275 - (view) Author: Robert Pierce (Robert Pierce) Date: 2020-02-03 09:56
It appears as if the problem is the sentinel FIFO opened by (for example) multiprocessing.popen_fork.Popen._launch(). It registers a finalization class to close the sentinel on garbage collection. Instead, it should be closed in poll() or wait() when the child process is reaped and known to be dead. The sentinel serves no purpose after the child is reaped, and waiting till garbage collection means that programs forking large numbers of processes cannot control file descriptor utilization.
History
Date User Action Args
2022-04-11 14:59:26adminsetgithub: 83716
2020-02-03 09:56:03Robert Piercesetmessages: + msg361275
2020-02-03 09:21:12Robert Piercecreate