diff --git a/Doc/library/markup.rst b/Doc/library/markup.rst --- a/Doc/library/markup.rst +++ b/Doc/library/markup.rst @@ -25,6 +25,7 @@ htmlparser.rst sgmllib.rst htmllib.rst + xml.rst xml.etree.elementtree.rst xml.dom.rst xml.dom.minidom.rst diff --git a/Doc/library/xml.dom.minidom.rst b/Doc/library/xml.dom.minidom.rst --- a/Doc/library/xml.dom.minidom.rst +++ b/Doc/library/xml.dom.minidom.rst @@ -20,6 +20,14 @@ not already proficient with the DOM should consider using the :mod:`xml.etree.ElementTree` module for their XML processing instead + +.. warning:: + + The :mod:`xml.dom.minidom` module is not secure against erroneous or + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see :ref:`xml-vulnerabilities`. + + DOM applications typically start by parsing some XML into a DOM. With :mod:`xml.dom.minidom`, this is done through the parse functions:: diff --git a/Doc/library/xml.dom.pulldom.rst b/Doc/library/xml.dom.pulldom.rst --- a/Doc/library/xml.dom.pulldom.rst +++ b/Doc/library/xml.dom.pulldom.rst @@ -16,6 +16,13 @@ Object Model representation of a document from SAX events. +.. warning:: + + The :mod:`xml.dom.pulldom` module is not secure against erroneous or + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see :ref:`xml-vulnerabilities`. + + .. class:: PullDOM([documentFactory]) :class:`xml.sax.handler.ContentHandler` implementation that ... diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -16,6 +16,14 @@ hierarchical data structures in memory. The type can be described as a cross between a list and a dictionary. + +.. warning:: + + The :mod:`xml.etree.ElementTree` module is not secure against erroneous or + maliciously constructed data. If you need to parse untrusted or + unauthenticated data see :ref:`xml-vulnerabilities`. + + Each element has a number of properties associated with it: * a tag which is a string identifying what kind of data this element represents diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst new file mode 100644 --- /dev/null +++ b/Doc/library/xml.rst @@ -0,0 +1,131 @@ +.. _xml: + +XML Processing Modules +====================== + +.. module:: xml + :synopsis: Package containing XML processing modules +.. sectionauthor:: Christian Heimes +.. sectionauthor:: Georg Brandl + + +Python's interfaces for processing XML are grouped in the ``xml`` package. + +.. warning:: + + The XML modules are not secure against erroneous or maliciously + constructed data. If you need to parse untrusted or unauthenticated data see + :ref:`xml-vulnerabilities`. + +It is important to note that modules in the :mod:`xml` package require that +there be at least one SAX-compliant XML parser available. The Expat parser is +included with Python, so the :mod:`xml.parsers.expat` module will always be +available. + +The documentation for the :mod:`xml.dom` and :mod:`xml.sax` packages are the +definition of the Python bindings for the DOM and SAX interfaces. + +The XML handling submodules are: + +* :mod:`xml.etree.ElementTree`: the ElementTree API, a simple and lightweight + +.. + +* :mod:`xml.dom`: the DOM API definition +* :mod:`xml.dom.minidom`: a lightweight DOM implementation +* :mod:`xml.dom.pulldom`: support for building partial DOM trees + +.. + +* :mod:`xml.sax`: SAX2 base classes and convenience functions +* :mod:`xml.parsers.expat`: the Expat parser binding + + +.. _xml-vulnerabilities: + +XML vulnerabilities +=================== + +The XML processing modules are not secure against maliciously constructed data. +An attacker can abuse vulnerabilities for e.g. denial of service attacks, to +access local files, to generate network connections to other machines, or +to or circumvent firewalls. The attacks on XML abuse unfamiliar features +like inline `DTD`_ (document type definition) with entities. + + +========================= ======== ========= ========= ======== ========= +kind sax etree minidom pulldom xmlrpc +========================= ======== ========= ========= ======== ========= +billion laughs **True** **True** **True** **True** **True** +quadratic blowup **True** **True** **True** **True** **True** +external entity expansion **True** False (1) False (2) **True** False (3) +DTD retrieval **True** False False **True** False +decompression bomb False False False False **True** +========================= ======== ========= ========= ======== ========= + +1. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a + ParserError when an entity occurs. +2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns + the unexpanded entity verbatim. +3. :mod:`xmlrpclib` doesn't expand external entities and omits them. + + +billion laughs / exponential entity expansion + The `Billion Laughs`_ attack -- also known as exponential entity expansion -- + uses multiple levels of nested entities. Each entity refers to another entity + several times, the final entity definition contains a small string. Eventually + the small string is expanded to several gigabytes. The exponential expansion + consumes lots of CPU time, too. + +quadratic blowup entity expansion + A quadratic blowup attack is similar to a `Billion Laughs`_ attack; it abuses + entity expansion, too. Instead of nested entities it repeats one large entity + with a couple of thousand chars over and over again. The attack isn't as + efficient as the exponential case but it avoids triggering countermeasures of + parsers against heavily nested entities. + +external entity expansion + Entity declarations can contain more than just text for replacement. They can + also point to external resources by public identifiers or system identifiers. + System identifiers are standard URIs or can refer to local files. The XML + parser retrieves the resource with e.g. HTTP or FTP requests and embeds the + content into the XML document. + +DTD retrieval + Some XML libraries like Python's mod:'xml.dom.pulldom' retrieve document type + definitions from remote or local locations. The feature has similar + implications as the external entity expansion issue. + +decompression bomb + The issue of decompression bombs (aka `ZIP bomb`_) apply to all XML libraries + that can parse compressed XML stream like gzipped HTTP streams or LZMA-ed + files. For an attacker it can reduce the amount of transmitted data by three + magnitudes or more. + +The documentation of `defusedxml`_ on PyPI has further information about +all known attack vectors with examples and references. + +defused packages +---------------- + +`defusedxml`_ is a pure Python package with modified subclasses of all stdlib +XML parsers that prevent any potentially malicious operation. The courses of +action are recommended for any server code that parses untrusted XML data. The +package also ships with example exploits and an extended documentation on more +XML exploits like xpath injection. + +`defusedexpat`_ provides a modified libexpat and patched replacment +:mod:`pyexpat` extension module with countermeasures against entity expansion +DoS attacks. Defusedexpat still allows a sane and configurable amount of entity +expansions. The modifications will be merged into future releases of Python. + +The workarounds and modifications are not included in patch releases as they +break backward compatibility. After all inline DTD and entity expansion are +well-definied XML features. + + +.. _defusedxml: +.. _defusedexpat: +.. _Billion Laughs: http://en.wikipedia.org/wiki/Billion_laughs +.. _ZIP bomb: http://en.wikipedia.org/wiki/Zip_bomb +.. _DTD: http://en.wikipedia.org/wiki/Document_Type_Definition diff --git a/Doc/library/xml.sax.rst b/Doc/library/xml.sax.rst --- a/Doc/library/xml.sax.rst +++ b/Doc/library/xml.sax.rst @@ -16,6 +16,14 @@ SAX exceptions and the convenience functions which will be most used by users of the SAX API. + +.. warning:: + + The :mod:`xml.sax` module is not secure against erroneous or maliciously + constructed data. If you need to parse untrusted or unauthenticated data see + :ref:`xml-vulnerabilities`. + + The convenience functions are: diff --git a/Doc/library/xmlrpclib.rst b/Doc/library/xmlrpclib.rst --- a/Doc/library/xmlrpclib.rst +++ b/Doc/library/xmlrpclib.rst @@ -28,6 +28,13 @@ between conformable Python objects and XML on the wire. +.. warning:: + + The :mod:`xmlrpclib` module is not secure against erroneous or maliciously + constructed data. If you need to parse untrusted or unauthenticated data see + :ref:`xml-vulnerabilities`. + + .. class:: ServerProxy(uri[, transport[, encoding[, verbose[, allow_none[, use_datetime]]]]]) A :class:`ServerProxy` instance is an object that manages communication with a