This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author asmodai
Recipients asmodai, rhettinger
Date 2008-03-06.19:39:27
SpamBayes Score 0.0003557684
Marked as misclassified No
Message-id <1204832369.47.0.680636098879.issue2246@psf.upfronthosting.co.za>
In-reply-to
Content
Quoting from my email to Raymond:

In the Trac/Genshi community we've been tracking a bit obscure memory 
leak that causes us a lot of problems.

Please see http://trac.edgewall.org/ticket/6614 and then
http://genshi.edgewall.org/ticket/190 for background.

We reduced the case to the following Python only code and believe it is 
a bug within itertool's groupby. As Armin Ronacher explains in Genshi 
ticket 190:

"Looks like genshi is not to blame. itertools.groupby has a grouper 
with a reference to the groupby type but no traverse func. As soon as a 
circular reference ends up in the groupby (which happens thanks to the 
func_globals in our lambda) genshi leaks."

This can be demonstrated with the following code (testcase attachment 
present with this issue):

import gc
from itertools import groupby

def run():
    keyfunc = lambda x: x
    for i, j in groupby(range(100), key=keyfunc):
        keyfunc.x = j

for x in xrange(20):
    gc.collect()
    run()
    print len(gc.get_objects())

On executing this in will show numerical output of the garbage 
collector, but every iteration will be +4 from the previous, as Armin 
specifies:

  "a frame, a grouper, a keyfunc and a groupby object"

We have been unable to come up with a decent patch and thus I am 
logging this issue now.
History
Date User Action Args
2008-03-06 19:39:29asmodaisetspambayes_score: 0.000355768 -> 0.0003557684
recipients: + asmodai, rhettinger
2008-03-06 19:39:29asmodaisetspambayes_score: 0.000355768 -> 0.000355768
messageid: <1204832369.47.0.680636098879.issue2246@psf.upfronthosting.co.za>
2008-03-06 19:39:28asmodailinkissue2246 messages
2008-03-06 19:39:28asmodaicreate