Author sjmachin
Recipients akitada, akuchling, amaury.forgeotdarc, collinwinter, doerwalter, ezio.melotti, georg.brandl, gregory.p.smith, jaylogan, jimjjewett, loewis, mark, moreati, mrabarnett, nneonneo, pitrou, rsc, sjmachin, timehorse
Date 2009-08-03.22:36:34
SpamBayes Score 3.60614e-11
Marked as misclassified No
Message-id <1249338995.73.0.429285017679.issue2636@psf.upfronthosting.co.za>
In-reply-to
Content
Problem is memory leak from repeated calls of e.g.
compiled_pattern.search(some_text). Task Manager performance panel shows
increasing memory usage with regex but not with re. It appears to be
cumulative i.e. changing to another pattern or text doesn't release memory.

Environment: Python 2.6.2, Windows XP SP3, latest (29 July) regex zip file.

Example:

8<-- regex_timer.py
import sys
import time
if sys.platform == 'win32':
    timer = time.clock
else:
    timer = time.time
module = __import__(sys.argv[1])
count = int(sys.argv[2])
pattern = sys.argv[3]
expected = sys.argv[4]
text = 80 * '~' + 'qwerty'
rx = module.compile(pattern)
t0 = timer()
for i in xrange(count):
    assert rx.search(text).group(0) == expected
t1 = timer()
print "%d iterations in %.6f seconds" % (count, t1 - t0)
8<---

Here are the results of running this (plus observed difference between
peak memory usage and base memory usage):

dos-prompt>\python26\python regex_timer.py regex 1000000 "~" "~"
1000000 iterations in 3.811500 seconds [60 Mb]

dos-prompt>\python26\python regex_timer.py regex 2000000 "~" "~"
2000000 iterations in 7.581335 seconds [128 Mb]

dos-prompt>\python26\python regex_timer.py re 2000000 "~" "~"
2000000 iterations in 2.549738 seconds [3 Mb]

This happens on a variety of patterns: "w", "wert", "[a-z]+", "[a-z]+t",
...
History
Date User Action Args
2009-08-03 22:36:36sjmachinsetrecipients: + sjmachin, loewis, akuchling, doerwalter, georg.brandl, collinwinter, gregory.p.smith, jimjjewett, amaury.forgeotdarc, pitrou, nneonneo, rsc, timehorse, mark, ezio.melotti, mrabarnett, jaylogan, akitada, moreati
2009-08-03 22:36:35sjmachinsetmessageid: <1249338995.73.0.429285017679.issue2636@psf.upfronthosting.co.za>
2009-08-03 22:36:34sjmachinlinkissue2636 messages
2009-08-03 22:36:34sjmachincreate