This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author jnoller
Recipients jnoller
Date 2009-08-05.20:25:01
SpamBayes Score 2.3022264e-09
Marked as misclassified No
Message-id <1249503902.75.0.475532249701.issue6653@psf.upfronthosting.co.za>
In-reply-to
Content
Additional comments from Sturla:

Hello Jesse,

Yes there is a bug in multiprocessing.

Diagnosis:

- Processes created by multiprocessing (mp.Process or mp.Pool) exit in a 
way that prevents the Python interpreter from running deallocation code 
for all extension objects (only the locals are cleaned up). Resources 
allocated by extension objects referenced in globals may leak 
permanently.

Sub-processes seem to commit an ungraceful suicide on exit. If the 
kernel cleans up after a non-graceful exit this is ok. But if the kernel 
do not, as in the case of System V IPC objects, we have a permanent 
resource leak. This is very similar to the reason why manually killing 
threads is prohibited in Python.

I have example code to show this. It creates a system-wide memory leak 
on Linux/Unix (present until the next reboot), unless the last statement 
in the target of mp.Process ensures a manual clean up of the globals.
History
Date User Action Args
2009-08-05 20:25:02jnollersetrecipients: + jnoller
2009-08-05 20:25:02jnollersetmessageid: <1249503902.75.0.475532249701.issue6653@psf.upfronthosting.co.za>
2009-08-05 20:25:01jnollerlinkissue6653 messages
2009-08-05 20:25:01jnollercreate