This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: memory leak in email.generator.Generator().flatten() method
Type: resource usage Stage: resolved
Components: IO, Library (Lib) Versions: Python 2.6
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Kaushik.Kannan, amaury.forgeotdarc, barry, kauboy, r.david.murray, terry.reedy
Priority: normal Keywords:

Created on 2011-03-27 08:25 by Kaushik.Kannan, last changed 2022-04-11 14:57 by admin. This issue is now closed.

File name Uploaded Description Edit Kaushik.Kannan, 2011-03-27 08:25 Running the attached python code leaks memory during the flatten() method call.
MemLeak.log kauboy, 2011-03-28 08:58
Messages (5)
msg132387 - (view) Author: Kaushik Kannan (kauboy) Date: 2011-03-28 08:54
I wrote a daemon to monitor a directory and send out emails periodically. To create the email, I used MIMEMultipart() object. When as_string() method is called on the MIMEMultipart() object, it seemed to cause memory leaks. On looking at the as_string() method, I saw that the  email.generator.Generator().flatten() method call is causing the memory leak. I copied the as_string() method out as a function and used it for tracing the memory leak.


from guppy import hpy
import time
import gc

from email.MIMEMultipart import MIMEMultipart
from email.MIMEText import MIMEText
from cStringIO import StringIO
from email.generator import Generator

def as_string_mod(msg, unixfrom=False):
        """Return the entire formatted message as a string.
        Optional `unixfrom' when True, means include the Unix From_ envelope

        This is a convenience method and may not generate the message exactly
        as you intend because by default it mangles lines that begin with
        "From ".  For more flexibility, use the flatten() method of a
        Generator instance.
        fp = StringIO()
        g = Generator(fp)
        g.flatten(msg, unixfrom=unixfrom)
        return fp.getvalue()

msg = MIMEMultipart()
msg['From'] = ''
msg['To'] = ''
msg['Subject'] = 'Function'

if __name__=='__main__':
	while True:
		print hpy().heap()
msg132388 - (view) Author: Kaushik Kannan (kauboy) Date: 2011-03-28 08:58
The memory usage trace as printed out by guppy is attached. For every new call of the Generator.flatten() method by the MIMEMultipart.as_string() method, the number of objects increases by 3 and the total size increases by 484Bytes.
msg132408 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2011-03-28 17:47
I replaced guppy by a debug build of Python and a simple "print sys.gettotalrefcount()".
Indeed, I see a reference leak with Python2.6 (10 more references on every loop). There seems to be no leak with Python2.7.
Which version of Python do you use?
msg132451 - (view) Author: Kaushik Kannan (kauboy) Date: 2011-03-29 00:45
I'm on Ubuntu 10.04 with Python 2.6.5, Python 3.1 is also available in the repos, I'll try that and post back.
msg228810 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-10-08 19:15
From Anaury's report, this is fixed in 2.7.  Email in current 3.x has been re-written.
Date User Action Args
2022-04-11 14:57:15adminsetgithub: 55902
2014-10-08 19:15:18terry.reedysetstatus: open -> closed

nosy: + terry.reedy
messages: + msg228810

resolution: fixed
stage: resolved
2011-03-29 00:45:14kauboysetmessages: + msg132451
2011-03-28 17:47:41amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg132408
2011-03-28 08:58:01kauboysetfiles: + MemLeak.log

messages: + msg132388
2011-03-28 08:54:04kauboysetnosy: + kauboy
messages: + msg132387
2011-03-27 08:25:27Kaushik.Kannancreate