Author serhiy.storchaka
Recipients ezio.melotti, mrabarnett, serhiy.storchaka
Date 2013-08-08.13:10:14
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1375967417.95.0.0303828296889.issue18685@psf.upfronthosting.co.za>
In-reply-to
Content
Before PEP 393 the regex functions scanned an array of char or Py_UNICODE and character testing was cheap. After PEP 393 they checks a kind of an unicode string for every tested character and processing of unicode strings becomes slower. _sre.c already generates two sets of functions from one source -- for byte and unicode strings. The proposed patch uses same technique to generate three sets of functions -- for byte/UCS1, UCS2 and UCS4 strings. This simplifies the code (now it more similar to pre-PEP393 version) and makes characters testing faster.

Benchmark example:

Python 3.2:
$ python3.2 -m timeit -s "import re; f = re.compile(b'abc').search; x = b'x'*100000"  "f(x)"
1000 loops, best of 3: 613 usec per loop
$ python3.2 -m timeit -s "import re; f = re.compile('abc').search; x = 'x'*100000"  "f(x)"
1000 loops, best of 3: 232 usec per loop
$ python3.2 -m timeit -s "import re; f = re.compile('abc').search; x = '\u20ac'*100000"  "f(x)"
1000 loops, best of 3: 217 usec per loop

Python 3.4.0a1+ unpatched:
$ ./python -m timeit -s "import re; f = re.compile(b'abc').search; x = b'x'*100000"  "f(x)"
1000 loops, best of 3: 485 usec per loop
$ ./python -m timeit -s "import re; f = re.compile('abc').search; x = 'x'*100000"  "f(x)"
1000 loops, best of 3: 790 usec per loop
$ ./python -m timeit -s "import re; f = re.compile('abc').search; x = '\u20ac'*100000"  "f(x)"
1000 loops, best of 3: 1.09 msec per loop

Python 3.4.0a1+ patched:
$ ./python -m timeit -s "import re; f = re.compile(b'abc').search; x = b'x'*100000"  "f(x)"
1000 loops, best of 3: 250 usec per loop
$ ./python -m timeit -s "import re; f = re.compile('abc').search; x = 'x'*100000"  "f(x)"
1000 loops, best of 3: 250 usec per loop
$ ./python -m timeit -s "import re; f = re.compile('abc').search; x = '\u20ac'*100000"  "f(x)"
1000 loops, best of 3: 256 usec per loop

I also propose for simplicity extract a template part of _sre.c to separated file (i.e. srelib.h) and get rid of recursion.
History
Date User Action Args
2013-08-08 13:10:19serhiy.storchakasetrecipients: + serhiy.storchaka, ezio.melotti, mrabarnett
2013-08-08 13:10:17serhiy.storchakasetmessageid: <1375967417.95.0.0303828296889.issue18685@psf.upfronthosting.co.za>
2013-08-08 13:10:17serhiy.storchakalinkissue18685 messages
2013-08-08 13:10:17serhiy.storchakacreate