classification
Title: sre_parse group limit check missing with 'python -O'
Type: Stage:
Components: Regular Expressions Versions: Python 2.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: effbot Nosy List: effbot, kbriggs, rushing (3)
Priority: normal Keywords

Created on 2004-11-12 20:46 by rushing, last changed 2004-12-21 16:23 by kbriggs.

Messages (3)
msg23114 - (view) Author: Sam Rushing (rushing) Date: 2004-11-12 20:46
sre_parse.py includes a check to ensure that the
hard-coded limit of 100 groups is not exceeded.
Since the check uses an assert(), it's not present
when running in '-O' mode, so you can easily segfault
the re engine:

rushing@fang:~$ /usr/local/bin/python -O
Python 2.3.4 (#2, Oct 23 2004, 05:24:36) 
[GCC 3.4.2 [FreeBSD] 20040728] on freebsd5
>>> import re
>>> p = re.compile ('(' + '|'.join (['(xxx)']*200) + ')')
>>> p.search ('asdfasdfasdfasdfasdf')
Segmentation fault (core dumped)

The assert() should be changed to an 'if/raise'.

NOTE: I've sent changes to Frederik to remove the
limitation altogether - so if those get in before this
bug is addressed then please ignore.
msg23115 - (view) Author: Fredrik Lundh (effbot) Date: 2004-11-13 07:02
Logged In: YES 
user_id=38376

This was changed in 2.4b1 (didn't I mail you about this?):

>>> p = re.compile ('(' + '|'.join (['(xxx)']*200) + ')')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/fredrik/NOBACKUP/python-
dev/python/dist/src/Lib/sre.py", line 180, in compile
    return _compile(pattern, flags)
  File "/home/fredrik/NOBACKUP/python-
dev/python/dist/src/Lib/sre.py", line 225, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/home/fredrik/NOBACKUP/python-
dev/python/dist/src/Lib/sre_compile.py", line 506, in compile
    raise AssertionError(
AssertionError: sorry, but this version only supports 100 
named groups

As for the real fix, it arrived too close to the beta release, 
and while it looks pretty solid, I'm not sure if it's a good idea 
to add it this close to a major release.  I'll bring this up on 
python-dev, when I find the time.
msg23116 - (view) Author: Keith Briggs (kbriggs) Date: 2004-12-21 16:23
Logged In: YES 
user_id=888261

There is a further problem here - the error message refers 
to "100 named groups", whereas the exception seems to be 
raised when there are too many groups, whether they are 
named or not.

What is the reason for this limit?  Can it easily be removed?
It is causing me many problems.
History
Date User Action Args
2004-11-12 20:46:23rushingcreate