classification
Title: test_bigmem.test_from_2G_generator uses more memory than expected
Type: resource usage Stage:
Components: Tests Versions: Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: twouters Nosy List: BreamoreBoy, pitrou, twouters
Priority: normal Keywords:

Created on 2009-03-08 01:25 by pitrou, last changed 2011-03-25 09:31 by twouters.

Messages (3)
msg83295 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-08 01:25
It seems test_from_2G_generator doesn't respect its announced memory
consumption. I launched test_bigmem with "-M 12G" on a 16GB machine, and
the interpreter process got killed by the OOM killer (Out Of Memory)
while running this test case.
msg109768 - (view) Author: Mark Lawrence (BreamoreBoy) Date: 2010-07-09 16:40
Is this problem repeatable?  I haven't got a big enough box on which to try this out.
msg132074 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2011-03-25 09:31
I'm not surprised this test takes up more memory... It claims to take up 9 times the test size in memory, with a minimum useful test size of 1G. With a limit of 12G that would make the test size about 1.33G. However, it then proceeds to build a tuple of 159 million distinct integers and stuffs them into a tuple. Each integer is at least 24 bytes long, plus the 8 of the pointer in the tuple, making the total memory use over 32 times the test size. I'll fix the test.
History
Date User Action Args
2011-03-25 09:31:41twouterssetversions: + Python 3.3, - Python 3.1, Python 2.7, Python 3.2
nosy: + twouters

messages: + msg132074

assignee: twouters
2010-07-09 16:40:04BreamoreBoysetnosy: + BreamoreBoy

messages: + msg109768
versions: + Python 2.7, Python 3.2
2009-03-08 01:25:13pitroucreate