Python XML parser (xml.etree) does not seems to allow control characters that are invalid in XML 1.0, but valid in XML 1.1 [1] [2]


Considering the following sample:


import xml.etree.ElementTree as ET

bad = '<?xml version="1.1"?><foo>bar &#x19; baz</foo>'
print(ET.fromstring(bad))


The parser raises the following error:
ParseError: reference to invalid character number: line 1, column 30



[1] https://www.w3.org/TR/xml11/Overview.html#charsets
[2] https://www.w3.org/TR/xml11/Overview.html#sec-xml11

It is a known issue, see issue11804 and issue39512.

In short, the underlying library for XML parsing (expat) does not support XML 1.1 and does not have plans to support it. And seems that XML 1.1 is a dead standard if it is not supported in popular parsing libraries.

From where you get your XML data? What programs generated them?

Thanks for the quick reply

We're getting data from about a hundred different providers around the world; some of them not really keen on standards, so we already have some hacks to fix invalid XML. We'll add one to the list

In that particular case, the XML was invalid anyway since it was an XML 1.0 document, and the character was sent as "binary" (\x19)