Message277015
Test-2.py has issues:
* it doesn't call Timer.join()
* it uses a weak synchronization between the main thread and the Timer thread: see msg276990 for an example using Event
If you use a better synchronization code, call timer.join() and call gc.collect(), the memory usage is very stable even after creating more than 100 000 threads + subprocesses.
Xavion: "As I noted before the problem is not with subprocess leaking 4K memory *always*. The issue comes from the fact that subprocess seems to leak 4K memory per individual thread."
I'm unable to reproduce a *leak*.
Test-3.py output:
----
497 thread+subprocess
VmRSS: 9584 kB
986 thread+subprocess
VmRSS: 9596 kB <== after the warmup, the usage seems stable
1490 thread+subprocess
VmRSS: 9596 kB
(...)
10361 thread+subprocess
VmRSS: 9596 kB
(...)
30282 thread+subprocess
VmRSS: 9596 kB
30695 thread+subprocess
VmRSS: 9672 kB
31160 thread+subprocess
VmRSS: 9684 kB <=== memory usage decreases :-)
(...)
60768 thread+subprocess
VmRSS: 9684 kB
^C
---
If you really want to say that something is wrong: I don't understand why we must call gc.collect() to keep the memory usage stable. But I guess that the GC is not always called for performance.
Without the GC it's not that bad:
---
1083 thread+subprocess
VmRSS: 9764 kB
2097 thread+subprocess
VmRSS: 9888 kB
3136 thread+subprocess
VmRSS: 9888 kB
(...)
11750 thread+subprocess
VmRSS: 9888 kB
12668 thread+subprocess
VmRSS: 9940 kB
13705 thread+subprocess
VmRSS: 9940 kB
(...)
70948 thread+subprocess
VmRSS: 9940 kB
^C
---
There is no such "4k leak per function call".
I close the issue. It's a bug in your code, not in Python. |
|
Date |
User |
Action |
Args |
2016-09-20 07:43:53 | vstinner | set | recipients:
+ vstinner, r.david.murray, ztane, The Compiler, Xavion |
2016-09-20 07:43:53 | vstinner | set | messageid: <1474357433.05.0.895180277395.issue28165@psf.upfronthosting.co.za> |
2016-09-20 07:43:53 | vstinner | link | issue28165 messages |
2016-09-20 07:43:52 | vstinner | create | |
|