This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: pprint ignores the compact parameter for dicts
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Nicolas Hug, docs@python, iansedano, iritkatriel, miss-islington, posita, serhiy.storchaka, xtreak
Priority: normal Keywords: patch

Created on 2018-09-25 14:45 by Nicolas Hug, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26967 merged iritkatriel, 2021-06-30 13:25
PR 26990 merged miss-islington, 2021-07-02 09:42
Messages (14)
msg326358 - (view) Author: Nicolas Hug (Nicolas Hug) Date: 2018-09-25 14:45
Dict representations that exceed the line width are printed with one line per key-value pair, ignoring the compact=True parameter:

>>> pprint.pprint({i: 0 for i in range(15)}, compact=True)
{0: 0,
 1: 0,
 2: 0,
 3: 0,
 4: 0,
 5: 0,
 6: 0,
 7: 0,
 8: 0,
 9: 0,
 10: 0,
 11: 0,
 12: 0,
 13: 0,
 14: 0}

Expected behavior:

>>> pprint.pprint({i: 0 for i in range(15)}, compact=True)
{0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0,
 12: 0, 13: 0, 14: 0}

Note that lists are correctly compacted, and that dicts that don't exceed line width are printed on a single line, regardless on the compact parameter.

I could try to work on that if needed?
msg326379 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-25 17:38
Thanks for the report Nicolas. I looked into the code and it seems that pprint for a dictionary now doesn't really take compact into account. List, sets, tuple and dequeue use _format_items [0] that honors value of compact but dictionary formatting uses _format_dict_items [1] that doesn't without using compact value. Unfortunately, I don't have any links over why dictionary doesn't take compact into account. I would suggest python-ideas [3] to get more feedback about formatting and implementation along with backwards compatibility so that you can proceed further. Your suggestion seems reasonable to me but I don't know if some program is using compact=True for a dictionary without knowing the internals in mind that might break for them. I would wait for others thoughts on this and I think this can be done only on 3.8 and not 3.7 which is in bug fix mode.

[0] https://github.com/python/cpython/blob/fdcb5ae25c0b5c82a32955617d253810ef110cac/Lib/pprint.py#L350
[1] https://github.com/python/cpython/blob/fdcb5ae25c0b5c82a32955617d253810ef110cac/Lib/pprint.py#L333
[2] https://mail.python.org/pipermail/python-ideas/

Hope this helps!
msg326385 - (view) Author: Nicolas Hug (Nicolas Hug) Date: 2018-09-25 18:27
Thank you for the feedback!

I'll try the python-ideas mail list. I posted a message on Python-list [1] a few weeks ago but it didn't get much traction.

I'm not sure about what the final solution could be (if any), but I had to hack pprint myself [2] for the scikit-learn project, and what I did was simply copy pasting _format_items into _format_dict_items, with some minimal changes.


[1] https://mail.python.org/pipermail/python-list/2018-September/737082.html
[2]
msg326386 - (view) Author: Nicolas Hug (Nicolas Hug) Date: 2018-09-25 18:29
Sorry:

[2] https://github.com/scikit-learn/scikit-learn/pull/11705/files#diff-f83e8d9362766b385472f1be7fed9482R96
msg326387 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-25 18:39
It is on purpose. See issue19132.
msg326391 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-25 18:51
Thanks much Serhiy for the details!
msg326394 - (view) Author: Nicolas Hug (Nicolas Hug) Date: 2018-09-25 19:01
Thanks for the link,

But I don't see any justification for this behavior*? Why should lists be compacted but not dicts (even when explicitly asked)?

At the very least it should be made clear in the documentation that dicts are not compacted.

* Maybe there's more content in the patch reviews but I am unable to see them
msg375805 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-08-22 22:51
> At the very least it should be made clear in the documentation that dicts are not compacted.

According to https://docs.python.org/3/library/pprint.html compact impacts the way that sequences are displayed, and a dict is not a sequence.
So I'm not sure a documentation change is required.
msg396777 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-30 13:27
I made PR26967 to break up the paragraph about the constructor params so that it's easier to read, and also added emphasis around the "sequence" point.
msg396851 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-07-02 09:42
New changeset 943e77d42d3f84b581f32c05f1fc8c05366b8ed3 by Irit Katriel in branch 'main':
bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… (GH-26967)
https://github.com/python/cpython/commit/943e77d42d3f84b581f32c05f1fc8c05366b8ed3
msg396852 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-07-02 10:08
New changeset 42c26282a123d14591e1aa31107e566b302a19ac by Miss Islington (bot) in branch '3.10':
bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… (GH-26967) (GH-26990)
https://github.com/python/cpython/commit/42c26282a123d14591e1aa31107e566b302a19ac
msg404952 - (view) Author: Ian Currie (iansedano) Date: 2021-10-25 08:19
I came across this and was confused by it too. I also don't understand the justification with not having dicts to be affected by the `compact` parameter.

If the "compact form" is having separate entries or elements on one line, instead of having each element separated by a new line, then it seems like inconsistent behavior.

**If a dict is short enough, it will appear in "compact form", just like a list.**

If a dict is too long for the width, then each item will appear in "expanded form", also like a list.
However, the actual compact parameter only affects sequence items. Why is this?

There is no reason given in #19132. It does mention a review, but it doesn't seem to be available, or I don't know how to get to it, to understand the reason for that decision.
msg408307 - (view) Author: Matt B (posita) * Date: 2021-12-11 14:34
Please consider highlighting that dicts are not included in the documentation. While *technically* true, this ...

> compact impacts the way that long sequences (lists, tuples, sets, etc) are formatted. If compact is false (the default) then each item of a sequence will be formatted on a separate line. If compact is true, as many items as will fit within the width will be formatted on each output line.

... has several problems.

First, sequence is a term of art that also has a common meaning. This creates a potential ambiguity in the understanding of the reader. Resolving that ambiguity in this context requires that readers have already internalized that dicts are not Python sequences. Those new to Python may not understand the (to them, subtle) differences between Python's iterables and sequences. Second, the "etc" only expands that ambiguity and invites confusion. Third, the term "items" is strongly associated with dicts and is used several times throughout the paragraph.

This ...

> According to https://docs.python.org/3/library/pprint.html compact impacts the way that sequences are displayed, and a dict is not a sequence.

... is unhelpfully pedantic, and ignorant of the needs of the newcomer (a key demographic of documentation). Documentation is a core product surface with a diverse audience. Rather than focus on technical correctness, documentation authors should focus on accessibility. Redundancy is a feature, not a bug. You can't predict how your reader got to that slice of the documentation. Imagine this as an opportunity to educate or reinforce concepts for readers, not as an opportunity to argue from a position of technical superiority.

The fact that there are now four developers who have taken their time to file patches, bugs, and comments is pretty strong signal that confusion exists among the audience and that the documentation is insufficient.
msg408317 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-11 16:41
Don't discuss on a closed issue. Create a new one if there is still a problem.
History
Date User Action Args
2022-04-11 14:59:06adminsetgithub: 78979
2021-12-11 16:41:58iritkatrielsetmessages: + msg408317
2021-12-11 14:34:42positasetnosy: + posita
messages: + msg408307
2021-10-25 08:19:23iansedanosetnosy: + iansedano
messages: + msg404952
2021-07-02 10:08:55iritkatrielsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-07-02 10:08:48iritkatrielsetversions: - Python 3.9
2021-07-02 10:08:07iritkatrielsetmessages: + msg396852
2021-07-02 09:42:39iritkatrielsetmessages: + msg396851
2021-07-02 09:42:21miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25552
2021-06-30 13:27:27iritkatrielsetmessages: + msg396777
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.8
2021-06-30 13:25:53iritkatrielsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25531
2020-09-19 18:57:39iritkatrielsetassignee: docs@python

components: + Documentation
nosy: + docs@python
2020-08-22 22:51:32iritkatrielsetnosy: + iritkatriel
messages: + msg375805
2018-09-25 19:01:11Nicolas Hugsetmessages: + msg326394
2018-09-25 18:51:35xtreaksetmessages: + msg326391
2018-09-25 18:39:45serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg326387
2018-09-25 18:29:21Nicolas Hugsetmessages: + msg326386
2018-09-25 18:27:54Nicolas Hugsetmessages: + msg326385
2018-09-25 18:20:56Nicolas Hugsetversions: + Python 3.8, - Python 3.7
2018-09-25 17:38:30xtreaksetmessages: + msg326379
2018-09-25 16:39:35xtreaksetnosy: + xtreak
2018-09-25 14:45:41Nicolas Hugcreate