classification
Title: multiprocessing leaks temporary directories pymp-xxx
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: sbt Nosy List: ZackerySpytz, sbt, vapier, yjhong
Priority: normal Keywords:

Created on 2014-06-04 18:39 by yjhong, last changed 2020-11-30 10:23 by iritkatriel. This issue is now closed.

Messages (3)
msg219767 - (view) Author: Yu-Ju Hong (yjhong) Date: 2014-06-04 18:39
When running many managers (e.g. > 10) in parallel on a decent machine, there is often a number of pymp-xxx directories left in /tmp after the run.

After some digging and debugging, I think the cause is that multiprocessing.managers.SyncManager waits for the manager process to shutdown and join with a timeout of 0.2s, and this timeout is way too low. This leads to processes being forced to terminate before cleaning up the temporary directories properly.

Could the timeout being raised a bit, or at least allowing the timeout to be set when creating the SyncManager through Manager()?
msg219770 - (view) Author: Mike Frysinger (vapier) Date: 2014-06-04 18:49
this has been fixed between py3.2 and py3.3:
http://hg.python.org/cpython/diff/831ae71d0bdc/Lib/multiprocessing/managers.py

so just asking for that to be backported to the py2.x branch :)
msg380633 - (view) Author: Zackery Spytz (ZackerySpytz) * (Python triager) Date: 2020-11-10 06:00
Python 2.7 is no longer supported, so I think this issue should be closed.
History
Date User Action Args
2020-11-30 10:23:04iritkatrielsetstatus: open -> closed
resolution: out of date
stage: resolved
2020-11-10 06:00:46ZackerySpytzsetnosy: + ZackerySpytz
messages: + msg380633
2014-06-28 18:01:19sbtsetassignee: sbt
2014-06-05 01:51:59ned.deilysetnosy: + sbt
2014-06-04 18:49:26vapiersetnosy: + vapier
messages: + msg219770
2014-06-04 18:39:04yjhongcreate