Author kevinbutler
Recipients
Date 2003-10-03.15:01:52
SpamBayes Score
Marked as misclassified
Message-id
In-reply-to
Content
The iterator returned by re.finditer appears to not
terminate if the 
final match is empty, but rather keeps returning the
final (empty) match.

Is this a bug in _sre?  If so, I'll be happy to file
it, though fixing 
it is a bit beyond my _sre experience level at this
point.  The solution 
would appear to be to either a check for duplicate
match in 
iterator.next(), or to increment position by one after
returning an 
empty match (which should be OK, because if a non-empty
match started at 
that location, we would have returned it instead of the
empty match).

Code to illustrate the failure:

from re import finditer

last = None
for m in finditer( ".*", "asdf" ):
    if last == m.span():
        print "duplicate match:", last
        break
    print m.group(), m.span()
    last = m.span()
   
---
asdf (0, 4)
 (4, 4)
duplicate match: (4, 4)
---

findall works:

print re.findall( ".*", "asdf" )
['asdf', '']

Workaround is to explicitly check for a duplicate span,
as I did above, 
or to check for a duplicate end(), which avoids the
final empty match

Seo Sanghyeon sent the following fix to python-dev list:

Attached one line patch fixes re.finditer bug reported by
Kevin J. Butler. I read cvs log to find out why this
code is
introduced, and it seems to be related to SF bug #581080.

But that bug didn't appear after my patch, so I wonder
why it was introduced in the first place. It seems beyond
my understanding. Please enlighten me.

To test:

#581080
import re
list(re.finditer('\s', 'a b'))
# expected: one item list
# bug: hang

#Kevin J. Butler
import re
list(re.finditer('.*', 'asdf'))
# expected: two item list (?)
# bug: hang

Seo Sanghyeon
-------------- next part --------------
? patch
Index: Modules/_sre.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/_sre.c,v
retrieving revision 2.99
diff -c -r2.99 _sre.c
*** Modules/_sre.c	26 Jun 2003 14:41:08 -0000	2.99
--- Modules/_sre.c	2 Oct 2003 03:48:55 -0000
***************
*** 3062,3069 ****
      match = pattern_new_match((PatternObject*)
self->pattern,
                                 state, status);
  
!     if ((status == 0 || state->ptr == state->start) &&
!         state->ptr < state->end)
          state->start = (void*) ((char*) state->ptr +
state->charsize);
      else
          state->start = state->ptr;
--- 3062,3068 ----
      match = pattern_new_match((PatternObject*)
self->pattern,
                                 state, status);
  
!     if (status == 0 || state->ptr == state->start)
          state->start = (void*) ((char*) state->ptr +
state->charsize);
      else
          state->start = state->ptr;
History
Date User Action Args
2007-08-23 14:17:27adminlinkissue817234 messages
2007-08-23 14:17:27admincreate