Title: fails with TypeError on invalid keyword in marked section
Type: Stage:
Components: Library (Lib) Versions: Python 3.8, Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: berker.peksag, bp256r1, ezio.melotti, kodial, leonardr
Priority: normal Keywords:

Created on 2019-08-02 18:38 by bp256r1, last changed 2020-06-11 20:23 by leonardr.

File name Uploaded Description Edit leonardr, 2020-06-11 20:23 Reproduce issue 37747 without using external packages
Messages (2)
msg348910 - (view) Author: bp256r1 (bp256r1) Date: 2019-08-02 18:38

I'm not sure if this a bug, but I noticed that a TypeError is raised by the parse_marked_section function of the _markupbase module in Python 3.7.4 when attempting to parse a name token of <![\r�N&=\x00%\x1a\x1e��;u�dWf'.


Steps to reproduce:

$ pip3 freeze | grep beautifulsoup4

$ python3
>>> a='<![\r�N&=\x00%\x1a\x1e��;u�dWf'
>>> from bs4 import BeautifulSoup
>>> BeautifulSoup(a, 'html.parser')
/usr/local/lib/python3.7/site-packages/bs4/builder/ UserWarning: expected name token at '<![\r�N&=\x00%\x1a\x1e��;u�dWf'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/bs4/", line 303, in __init__
  File "/usr/local/lib/python3.7/site-packages/bs4/", line 364, in _feed
  File "/usr/local/lib/python3.7/site-packages/bs4/builder/", line 250, in feed
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/", line 111, in feed
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/", line 179, in goahead
    k = self.parse_html_declaration(i)
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/html/", line 264, in parse_html_declaration
    return self.parse_marked_section(i)
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/", line 149, in parse_marked_section
    sectName, j = self._scan_name( i+3, i )
TypeError: cannot unpack non-iterable NoneType object

If it's not a bug, sorry, not sure.
msg371323 - (view) Author: Leonard Richardson (leonardr) * Date: 2020-06-11 20:23
This was also recently filed as a bug against Beautiful Soup, a package I maintain, using Python 3.8. (

The attached script reproduces the problem without using external packages.
Date User Action Args
2020-06-11 20:23:39leonardrsetfiles: +
versions: + Python 3.8
nosy: + leonardr

messages: + msg371323
2019-08-02 18:38:27bp256r1create