In Python 2.3.3, HTMLParser uses a certain regex that
is too stringent, and it does not capture slightly
malformed HTML gracefully.

The current definition of HTMLParser.locatestartendtag:

locatestarttagend = re.compile(r"""
  <[a-zA-Z][-.a-zA-Z0-9:_]*       # tag name
  (?:\s+                  # whitespace before attribute
name
    (?:[a-zA-Z_][-.:a-zA-Z0-9_]*     # attribute name
      (?:\s*=\s*                     # value indicator
        (?:'[^']*'                   # LITA-enclosed value
          |\"[^\"]*\"                # LIT-enclosed value
          |[^'\">\s]+                # bare value
         )
       )?
     )
   )*
  \s*                                # trailing whitespace
""", re.VERBOSE)


does not capture strings like:

    <IMG SRC = "abc.jpg"WIDTH=5>

where there is no space between the closing quote and
the next attribute name.  Many sources of HTML are
slightly malformed this way --- in particular, CNN.com
--- so being slightly lenient might be good.  We can
slightly relax the constraint:


locatestarttagend = re.compile(r"""
  <[a-zA-Z][-.a-zA-Z0-9:_]*       # tag name
  (?:\s*          # optional whitespace before
attribute name
    (?:[a-zA-Z_][-.:a-zA-Z0-9_]*     # attribute name
      (?:\s*=\s*                     # value indicator
        (?:'[^']*'                   # LITA-enclosed value
          |\"[^\"]*\"                # LIT-enclosed value
          |[^'\">\s]+                # bare value
         )
       )?
     )
   )*
  \s*                                # trailing whitespace
""", re.VERBOSE)

which allows the parser to process more of the HTML out
there.


See:

http://mail.python.org/pipermail/tutor/2004-October/032835.html

and:

http://mail.python.org/pipermail/tutor/2004-October/032869.html

for an explanation of what motivates this change.

Thanks!

The regex is still the same. This is one of many 'HTMLParser regex for
attributes' issues.