Issue227699
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2001-01-05 20:30 by cgw, last changed 2022-04-10 16:03 by admin. This issue is now closed.
Messages (8) | |||
---|---|---|---|
msg2826 - (view) | Author: Charles G Waldman (cgw) | Date: 2001-01-05 20:30 | |
#!/usr/bin/env python import os import sys import imp print os.getpid() sys.path.append("/tmp") count = 0 while (count<1000): modname = "testmod%s" % count count = count + 1 filename = '/tmp/' + modname + '.py' f = open(filename, 'w') for x in range(10000): f.write('x%s=%s\n'%(x,x)) f.close() modinfo = imp.find_module(modname) print 'loading', modname m = apply(imp.load_module, ('testmod',) + modinfo) ## run "watch -n 1 ps up <pid> ## where <pid> is the pid printed out by this program |
|||
msg2827 - (view) | Author: Fred Drake (fdrake) | Date: 2001-01-07 23:32 | |
Assigned to Barry since he's the memory leak expert (& has the right tools). |
|||
msg2828 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-01-19 03:43 | |
Barry...?!?! |
|||
msg2829 - (view) | Author: Barry A. Warsaw (barry) * | Date: 2001-02-23 19:53 | |
I can verify that memory grows with this process. I will run it under insure to get more information. |
|||
msg2830 - (view) | Author: Barry A. Warsaw (barry) * | Date: 2001-10-18 22:30 | |
Logged In: YES user_id=12800 I'm having a hard time tracking down the leak. I don't believe it's at the C layer since preliminary Insure runs don't indicate any leaked blocks (there are plenty of inuse blocks, but those generally don't count). I will note that if you comment out the imp.load_module call the memory usage remains constant. Loading each module though is going to naturally consume more memory, right? And those loaded modules aren't ever going to get freed, at least without mucking about in sys.modules. The memory consumption with ps is pretty meager, so I suspect we don't have a valid leak here, but just normal memory consumption due to importation of 10k modules. I'm running a more detailed insure run right now just to be sure, but I'm skeptical that there's really a bug here. |
|||
msg2831 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-10-18 23:28 | |
Logged In: YES user_id=6380 Barry, note that he's cheating with the module name passed to load_module(): the module name is always set to 'testmod', so he's in fact overwriting the module on each load. (This acts the same way as reload(), so it's not creating new module objects -- it's loading the code in the same namespace over and over.) I see the process size grow very quickly to 15M, probably on account of the huge parse tree for those 10,000 assignment statements (much less for the 10,000 actual variables created). After that it gradually grows -- after 100 loads it's up to 27 M. Maybe Insure doesn't see this because the data is actually kept alive somewhere? |
|||
msg2832 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-10-19 00:42 | |
Logged In: YES user_id=6380 Using the PYTHONDUMPREFS=1 environment variable and the debug version of Python, I don't see this leaking objects except that all the file names ('/tmp/testmod0.py' etc.) seem to be saved as interned strings, which eternalizes them of course. Hm, I wonder why the filename is interned? (I ran it twice, with different counts (100 and 1000), and 1000 instead of 10,000 variables, saving the PYTHONDUMPREFS output to two different files; then diffed the output files.) So if it's leaking beyond this, it's not in the form of objects. And note that Insure++ (or any leak detection tool worth the name) doesn't consider interned strings leaks -- they are reachable via a global variable. |
|||
msg2833 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2001-12-12 05:28 | |
Logged In: YES user_id=6380 Closing. The code base has changed so much that this is most likely irrelevant. Also, very recently Neal Norwitz ran Purify on the entire test suite and declared it leak-free. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-10 16:03:36 | admin | set | github: 33664 |
2001-01-05 20:30:55 | cgw | create |