classification
Title: Garbage collection not working correctly in Python 2.3
Type: resource usage Stage:
Components: Interpreter Core Versions: Python 2.3, Python 2.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: _doublep, christian.heimes, gvanrossum, pythonmeister
Priority: normal Keywords:

Created on 2007-11-08 16:31 by pythonmeister, last changed 2007-11-10 18:23 by gvanrossum. This issue is now closed.

Messages (7)
msg57256 - (view) Author: Stefan Sonnenberg-Carstens (pythonmeister) Date: 2007-11-08 16:31
when running this script:
aList = []
for i in xrange(5E5):
    aList += [[]]
    for j in xrange(10):
        aList[-1].append([])
del aList

It does not give back the memory

even a

import gc
gc.collect()

afterwards does not do it.

In Python 2.5 the memory is freed again correctly, at least under Windows.

The problem came up, because I was parsing a CSV file of 50 MB which
resulted in memory usage of more than 500 MB.
msg57263 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-08 17:30
I'm sorry but Python 2.3 is long gone. Its maintenance cycle has ended
over a year ago.Nobody is going to fix an outdated version when the new
versions of Python are working fine. Can you update to a new version of
Python?
msg57316 - (view) Author: Stefan Sonnenberg-Carstens (pythonmeister) Date: 2007-11-09 18:51
No, I can't.
As many Front Arena Developers on the 1.6/2.0/2.1/2.2 can't.
Python 2.4 will be in Front Arena 4.0.
Lightyears away from here.
Same behaviour seen under Solaris 10 / Python 2.5.1
msg57317 - (view) Author: Paul Pogonyshev (_doublep) Date: 2007-11-09 19:16
See if gc.set_threshold (0, 0, 0) helps.
msg57319 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-11-09 19:29
How do you know the memory isn't given back? It may be available for
reallocation within Python, just not given back to the operating system.
That's not necessarily a leak or a bug; that could just be heap
fragmentation. There's nothing you can do about it.
msg57351 - (view) Author: Paul Pogonyshev (_doublep) Date: 2007-11-10 15:34
Looks like the memory _is_ freed.  As Guido said, "It may be available
for reallocation within Python, just not given back to the operating
system".  I suggest closing this as invalid.

paul@gonzo:~$ python
Python 2.3.5 (#2, Oct 16 2006, 19:19:48)
[GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gc
>>> len ([object for object in gc.get_objects () if isinstance (object,
list)])
25
>>> aList = []
>>> for i in xrange(5E5):
...     aList += [[]]
...     for j in xrange(10):
...         aList[-1].append([]
...
...
KeyboardInterrupt
>>>         aList[-1].append([]
KeyboardInterrupt
>>>
paul@gonzo:~/emacs$ python
Python 2.3.5 (#2, Oct 16 2006, 19:19:48)
[GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gc
>>> len ([object for object in gc.get_objects () if isinstance (object,
list)])
25
>>> aList = []
>>> for i in xrange(5E5):
...     aList += [[]]
...     for j in xrange(10):
...         aList[-1].append([])
...
__main__:1: DeprecationWarning: integer argument expected, got float
>>> del aList
>>> len ([object for object in gc.get_objects () if isinstance (object,
list)])
25
msg57352 - (view) Author: Paul Pogonyshev (_doublep) Date: 2007-11-10 15:35
Meh, copied too much.  Disregard first part, second shows it.
History
Date User Action Args
2007-11-10 18:23:40gvanrossumsetstatus: open -> closed
resolution: not a bug
2007-11-10 15:35:38_doublepsetmessages: + msg57352
2007-11-10 15:34:38_doublepsetmessages: + msg57351
2007-11-09 19:29:22gvanrossumsetnosy: + gvanrossum
messages: + msg57319
2007-11-09 19:16:01_doublepsetnosy: + _doublep
messages: + msg57317
2007-11-09 18:56:04christian.heimessetstatus: closed -> open
resolution: out of date -> (no value)
2007-11-09 18:51:11pythonmeistersetmessages: + msg57316
versions: + Python 2.5
2007-11-08 17:30:00christian.heimessetstatus: open -> closed
resolution: out of date
messages: + msg57263
nosy: + christian.heimes
2007-11-08 16:31:30pythonmeistercreate