classification
Title: xml.etree.ElementTree: add feature to prettify XML output
Type: enhancement Stage: resolved
Components: Library (Lib), XML Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: scoder Nosy List: Andrew Grant, Clayton Olney, alex.dzyoba, alex.henderson, eli.bendersky, eric.araujo, eric.snow, loewis, martin.panter, mcepl, mitar, rhettinger, santoso.wijaya, scoder, serhiy.storchaka, tshepang, wolma
Priority: normal Keywords: patch

Created on 2012-04-01 15:28 by tshepang, last changed 2019-08-23 14:45 by scoder. This issue is now closed.

Files
File name Uploaded Description Edit
issue14465.patch alex.henderson, 2013-08-05 21:14 pretty printer patch, as implemented for issue 17372. review
Pull Requests
URL Status Linked Edit
PR 4016 closed alex.dzyoba, 2017-10-17 07:47
PR 8933 closed mcepl, 2018-08-25 19:22
PR 15200 merged scoder, 2019-08-10 17:06
Messages (16)
msg157299 - (view) Author: Tshepang Lekhonkhobe (tshepang) * Date: 2012-04-01 15:28
I often miss lxml's "pretty_print=True" functionality. Can you implement something similar.
msg157317 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-04-01 16:49
Would you like to provide a patch?
msg157320 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2012-04-01 17:59
Tshepang,

Frankly, there are a lot of issues to solve in ElementTree (it hasn't been given love in a long time...) and such features would be low priority, as I'm not getting much help and am swamped already.

As Martin said, patches can go a long way here...
msg157325 - (view) Author: Tshepang Lekhonkhobe (tshepang) * Date: 2012-04-01 19:08
Okay, I will try, even though C scares me.
msg157647 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-04-06 06:24
You may be able to code it entirely in the Python part of the module (adding a new parameter to Element.write and tostring).
msg194313 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2013-08-03 22:26
A patch exists in the duplicate #17372
msg194508 - (view) Author: Alex Henderson (alex.henderson) * Date: 2013-08-05 21:14
Proposed patch copied over from duplicate issue 17372.
msg194902 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2013-08-11 16:30
Just to reiterate this point, lxml.etree supports a "pretty_print" flag in its tostring() function and ElementTree.write(). It would thus make sense to support the same thing in ET.

http://lxml.de/api.html#serialisation

For completeness, the current signature looks like this:

def tostring(element_or_tree, *, encoding=None, method="xml",
             xml_declaration=None, pretty_print=False,
             with_tail=True, standalone=None, doctype=None,
             exclusive=False, with_comments=True,
             inclusive_ns_prefixes=None):

(The last three options are for C14N serialisation.)
msg304362 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-10-13 21:16
For the record, at 2015-04-02, the bpo-23847 has been marked as a duplicate of this issue.
msg304872 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-10-24 08:46
My thoughts:

1. Whitespaces are significant in XML. Pretty-printed XML is different from the original XML to an XML parser. For some applications some whitespaces around tags are not significant. But this depends on the application and in different parts of the document whitespaces can have different meaning. For example the document can contain a metadata with insignificant whitespaces and marked up text with significant whitespaces. There is a special attribute named xml:space that can signal the meaning of whitespaces for the part of a document.

https://www.w3.org/TR/xml/#sec-white-space

2. In HTML whitespaces around <P> are insignificant, but whitespaces around <I> are significant. Whitespaces inside <PRE>...</PRE> are significant.

3. If strip whitespaces around tags and insert newlines and indentations, shouldn't we strip whitespaces inside the text context? Or preserve newlines but update indentations?

4. If modify whitespaces on output, it may be worth to add an option to ignore insignificant whitespaces on input.

5. Serialization of ElementTree in the stdlib is much slower than in lxml (see issue25881). Perhaps it should be implemented in C. But it should be kept simple for this. Pretty-printing can be implemented as an outher preprocessing operation (for example the original Eli's code indents the tree in-place: http://effbot.org/zone/element-lib.htm#prettyprint) or as a proxy that indents elements on-fly.
msg323690 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2018-08-18 05:06
> Serialization of ElementTree in the stdlib is much slower than in lxml (see issue25881). Perhaps it should be implemented in C. But it should be kept simple for this.

Should I say it? That's a first class use case for Cython.


> Pretty-printing can be implemented as an outher preprocessing operation

Agreed. And that would actually be much simpler to implement in C.
msg324098 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-08-25 20:00
A few more thoughts for consideration:

* We already have a toprettyxml() tool in the minidom package.

* Since whitespace is significant in XML, prettifying changes the content and meaning, so it doesn't round-trip and should only be used for debugging purposes.

* Usually, I recommend using XML viewers such as the one built into the Chrome browser.  That provides indentation without changing meaning. It also lets you run searches and conveniently supports folding and unfolding elements.   I would rather someone use a viewer rather than something like toprettyxml().
msg335306 - (view) Author: Clayton Olney (Clayton Olney) Date: 2019-02-12 14:36
I have a use case where the receiving application is expecting the indentation, and I need to run my code in Lambda. So, lxml is out of the question.
msg349326 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-10 03:08
FWIW, here is the relevant section of the XML specification, https://www.w3.org/TR/2008/REC-xml-20081126/#sec-white-space :

"""In editing XML documents, it is often convenient to use "white space" (spaces, tabs, and blank lines) to set apart the markup for greater readability. Such white space is typically not intended for inclusion in the delivered version of the document. On the other hand, "significant" white space that should be preserved in the delivered version is common, for example in poetry and source code.

An XML processor must always pass all characters in a document that are not markup through to the application. A validating XML processor must also inform the application which of these characters constitute white space appearing in element content.
"""

OTOH, the java TransformerFactory does support a property, OutputKeys.INDENT, so there is a precedent for this feature request.

Stefan, would you please make a final determination or pronouncement on whether this makes sense for ElementTree or whether it is outside the scope of what the module is trying to accomplish.
msg349346 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2019-08-10 17:06
The spec section that Raymond quoted makes it clear that pretty printing is not for everyone. But there are many use cases where it is 1) helpful, 2) leads to correct results, and 3) does not grow the file size excessively. Whoever wants to make use of it is probably in such a situation. I think adding some kind of support in the standard library would be nice, but it should not hurt "normal" uses, especially when a lot of data is involved.

I'll send a PR that adds an indent() function to pre-process trees. Comments welcome.
msg350301 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2019-08-23 14:44
New changeset b5d3ceea48c181b3e2c6c67424317afed606bd39 by Stefan Behnel in branch 'master':
bpo-14465: Add an indent() function to xml.etree.ElementTree to pretty-print XML trees (GH-15200)
https://github.com/python/cpython/commit/b5d3ceea48c181b3e2c6c67424317afed606bd39
History
Date User Action Args
2019-08-23 14:45:28scodersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-08-23 14:44:37scodersetmessages: + msg350301
2019-08-12 13:07:08vstinnersetnosy: - vstinner
2019-08-10 17:06:37scodersetpull_requests: + pull_request14929
2019-08-10 17:06:16scodersetassignee: serhiy.storchaka -> scoder
messages: + msg349346
versions: + Python 3.9, - Python 3.7, Python 3.8
2019-08-10 03:08:27rhettingersetmessages: + msg349326
2019-08-08 15:53:22Andrew Grantsetnosy: + Andrew Grant
2019-06-22 20:31:20mitarsetnosy: + mitar
2019-02-12 14:36:01Clayton Olneysetnosy: + Clayton Olney
messages: + msg335306
2018-08-25 20:00:03rhettingersetnosy: + rhettinger
messages: + msg324098
2018-08-25 19:22:26mceplsetpull_requests: + pull_request8405
2018-08-18 05:06:24scodersetmessages: + msg323690
2018-08-17 15:23:10mceplsetversions: + Python 3.8
2017-10-24 08:46:06serhiy.storchakasetmessages: + msg304872
2017-10-22 20:32:54wolmasetnosy: + wolma
2017-10-18 12:19:04serhiy.storchakasetassignee: serhiy.storchaka

nosy: + serhiy.storchaka
components: + XML
versions: + Python 3.7, - Python 3.4
2017-10-17 07:47:12alex.dzyobasetstage: patch review
pull_requests: + pull_request3990
2017-10-14 17:31:29alex.dzyobasetnosy: + alex.dzyoba
2017-10-13 21:16:29vstinnersetnosy: + vstinner
messages: + msg304362
2015-04-02 08:20:47serhiy.storchakalinkissue23847 superseder
2015-01-27 10:23:02martin.pantersetnosy: + martin.panter
2015-01-27 09:26:05mceplsetnosy: + mcepl
2013-08-11 17:56:38eric.snowsetnosy: + eric.snow
2013-08-11 16:30:21scodersetnosy: + scoder

messages: + msg194902
versions: + Python 3.4, - Python 3.3
2013-08-05 21:14:48alex.hendersonsetfiles: + issue14465.patch

nosy: + alex.henderson
messages: + msg194508

keywords: + patch
2013-08-03 22:26:04eli.benderskysetmessages: + msg194313
2013-08-03 22:25:43eli.benderskylinkissue17372 superseder
2012-04-06 06:24:10eric.araujosetnosy: + eric.araujo
messages: + msg157647
2012-04-05 22:20:48santoso.wijayasetnosy: + santoso.wijaya
2012-04-01 19:08:10tshepangsetmessages: + msg157325
2012-04-01 17:59:20eli.benderskysetmessages: + msg157320
2012-04-01 16:49:32loewissetnosy: + loewis
messages: + msg157317
2012-04-01 16:43:46r.david.murraysettype: enhancement
2012-04-01 15:29:13tshepangsettitle: add feature to prettify XML output -> xml.etree.ElementTree: add feature to prettify XML output
2012-04-01 15:28:03tshepangcreate