Message91332
I have example code to show this. It creates a system-wide memory leak
on Linux/Unix (present until the next reboot), unless the last statement
in the target of mp.Process ensures a manual clean up of the globals.
The problem is line 353 in multiprocessing/forking.py. The function
exit() is defined as os._exit on Linux and ExitProcess on Windows, none
of which allows normal clean up.
>>> help(os._exit)
Help on built-in function _exit in module nt:
_exit(...)
_exit(status)
Exit to the system with specified status, without normal exit
processing.
The problem is fixed if line 353 in forking.py is changed from
exit(exitcode)
to
sys.exit(exitcode)
Test run without bugfix:
G:\DEVELO~1\SHARED~2>python test.py
open handle to 569f439b24e24fc8a547b81932616066
[[ 0. 0. 0. 0.]
[ 0. 0. 0. 0.]]
open handle to 0582d4b161c546f582c1c96e7bd0c39d
open handle to 569f439b24e24fc8a547b81932616066
modified array
closed handle to 569f439b24e24fc8a547b81932616066
[[ 1. 1. 1. 0.]
[ 1. 1. 1. 0.]]
closed handle to 569f439b24e24fc8a547b81932616066
You can see here that opening and closing of handles are unmatched. This
is on Windows, where the kernel ensures the clean-up, so it may not
matter. But on Unix this would have created a permament (system wide)
memory leak! What is happening here is globals not being cleaned up due
to the use of os._exit instead of sys.exit.
Test run with bugfix:
G:\DEVELO~1\SHARED~2>python test.py
open handle to 930778d27b414253bc329f2b70adaa05
[[ 0. 0. 0. 0.]
[ 0. 0. 0. 0.]]
open handle to 3f6cebf8c5de413685bb770d02ae9666
open handle to 930778d27b414253bc329f2b70adaa05
modified array
closed handle to 930778d27b414253bc329f2b70adaa05
closed handle to 3f6cebf8c5de413685bb770d02ae9666
[[ 1. 1. 1. 0.]
[ 1. 1. 1. 0.]]
closed handle to 930778d27b414253bc329f2b70adaa05
Now all allocations and deallocations are matched.
Regards,
Sturla Molden |
|
Date |
User |
Action |
Args |
2009-08-05 20:23:52 | jnoller | set | recipients:
+ jnoller |
2009-08-05 20:23:52 | jnoller | set | messageid: <1249503832.66.0.0114205396453.issue6653@psf.upfronthosting.co.za> |
2009-08-05 20:23:50 | jnoller | link | issue6653 messages |
2009-08-05 20:23:50 | jnoller | create | |
|