Message233067
Sorry for the additional followup, but I re-ran the test with approximately real-world parameters. In this second test, I created 10 million Foo() instances with random values from 1-2000 (using random.randint). This corresponds to "2000 hosts generating 10 million logs".
Memory use is ~80 MB. Times: CPU times: user 27.5 s, sys: 101 ms, total: 27.6 s Wall time: 27.6 s
For the nonoptimized run (10m instances of 2000 random values):
Memory use is ~1.8GB. Times: CPU times: user 28.2 s, sys: 1.29 s, total: 29.5 s Wall time: 29.5 s |
|
Date |
User |
Action |
Args |
2014-12-23 23:27:11 | sbromberger | set | recipients:
+ sbromberger, lemburg, rhettinger, ncoghlan, pmoody, r.david.murray, serhiy.storchaka, josh.r |
2014-12-23 23:27:11 | sbromberger | set | messageid: <1419377231.95.0.330190455135.issue23103@psf.upfronthosting.co.za> |
2014-12-23 23:27:11 | sbromberger | link | issue23103 messages |
2014-12-23 23:27:11 | sbromberger | create | |
|