Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pprint ignores the compact parameter for dicts #78979

Closed
NicolasHug mannequin opened this issue Sep 25, 2018 · 14 comments
Closed

pprint ignores the compact parameter for dicts #78979

NicolasHug mannequin opened this issue Sep 25, 2018 · 14 comments
Labels
3.10 only security fixes 3.11 only security fixes docs Documentation in the Doc dir type-bug An unexpected behavior, bug, or error

Comments

@NicolasHug
Copy link
Mannequin

NicolasHug mannequin commented Sep 25, 2018

BPO 34798
Nosy @serhiy-storchaka, @posita, @miss-islington, @tirkarthi, @iritkatriel, @iansedano
PRs
  • bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… #26967
  • [3.10] bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… (GH-26967) #26990
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-07-02.10:08:55.398>
    created_at = <Date 2018-09-25.14:45:41.804>
    labels = ['3.11', 'type-bug', '3.10', 'docs']
    title = 'pprint ignores the compact parameter for dicts'
    updated_at = <Date 2021-12-11.16:41:58.483>
    user = 'https://bugs.python.org/NicolasHug'

    bugs.python.org fields:

    activity = <Date 2021-12-11.16:41:58.483>
    actor = 'iritkatriel'
    assignee = 'docs@python'
    closed = True
    closed_date = <Date 2021-07-02.10:08:55.398>
    closer = 'iritkatriel'
    components = ['Documentation']
    creation = <Date 2018-09-25.14:45:41.804>
    creator = 'Nicolas Hug'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 34798
    keywords = ['patch']
    message_count = 14.0
    messages = ['326358', '326379', '326385', '326386', '326387', '326391', '326394', '375805', '396777', '396851', '396852', '404952', '408307', '408317']
    nosy_count = 8.0
    nosy_names = ['docs@python', 'serhiy.storchaka', 'posita', 'miss-islington', 'xtreak', 'Nicolas Hug', 'iritkatriel', 'iansedano']
    pr_nums = ['26967', '26990']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue34798'
    versions = ['Python 3.10', 'Python 3.11']

    @NicolasHug
    Copy link
    Mannequin Author

    NicolasHug mannequin commented Sep 25, 2018

    Dict representations that exceed the line width are printed with one line per key-value pair, ignoring the compact=True parameter:

    >>> pprint.pprint({i: 0 for i in range(15)}, compact=True)
    {0: 0,
     1: 0,
     2: 0,
     3: 0,
     4: 0,
     5: 0,
     6: 0,
     7: 0,
     8: 0,
     9: 0,
     10: 0,
     11: 0,
     12: 0,
     13: 0,
     14: 0}

    Expected behavior:

    >>> pprint.pprint({i: 0 for i in range(15)}, compact=True)
    {0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0,
     12: 0, 13: 0, 14: 0}

    Note that lists are correctly compacted, and that dicts that don't exceed line width are printed on a single line, regardless on the compact parameter.

    I could try to work on that if needed?

    @NicolasHug NicolasHug mannequin added 3.7 (EOL) end of life type-bug An unexpected behavior, bug, or error labels Sep 25, 2018
    @tirkarthi
    Copy link
    Member

    Thanks for the report Nicolas. I looked into the code and it seems that pprint for a dictionary now doesn't really take compact into account. List, sets, tuple and dequeue use _format_items [0] that honors value of compact but dictionary formatting uses _format_dict_items [1] that doesn't without using compact value. Unfortunately, I don't have any links over why dictionary doesn't take compact into account. I would suggest python-ideas [3] to get more feedback about formatting and implementation along with backwards compatibility so that you can proceed further. Your suggestion seems reasonable to me but I don't know if some program is using compact=True for a dictionary without knowing the internals in mind that might break for them. I would wait for others thoughts on this and I think this can be done only on 3.8 and not 3.7 which is in bug fix mode.

    [0]

    def _format_items(self, items, stream, indent, allowance, context, level):

    [1]
    def _format_dict_items(self, items, stream, indent, allowance, context,

    [2] https://mail.python.org/pipermail/python-ideas/

    Hope this helps!

    @NicolasHug NicolasHug mannequin added 3.8 only security fixes and removed 3.7 (EOL) end of life labels Sep 25, 2018
    @NicolasHug
    Copy link
    Mannequin Author

    NicolasHug mannequin commented Sep 25, 2018

    Thank you for the feedback!

    I'll try the python-ideas mail list. I posted a message on Python-list [1] a few weeks ago but it didn't get much traction.

    I'm not sure about what the final solution could be (if any), but I had to hack pprint myself [2] for the scikit-learn project, and what I did was simply copy pasting _format_items into _format_dict_items, with some minimal changes.

    [1] https://mail.python.org/pipermail/python-list/2018-September/737082.html
    [2]

    @NicolasHug
    Copy link
    Mannequin Author

    NicolasHug mannequin commented Sep 25, 2018

    @serhiy-storchaka
    Copy link
    Member

    It is on purpose. See bpo-19132.

    @tirkarthi
    Copy link
    Member

    Thanks much Serhiy for the details!

    @NicolasHug
    Copy link
    Mannequin Author

    NicolasHug mannequin commented Sep 25, 2018

    Thanks for the link,

    But I don't see any justification for this behavior*? Why should lists be compacted but not dicts (even when explicitly asked)?

    At the very least it should be made clear in the documentation that dicts are not compacted.

    • Maybe there's more content in the patch reviews but I am unable to see them

    @iritkatriel
    Copy link
    Member

    At the very least it should be made clear in the documentation that dicts are not compacted.

    According to https://docs.python.org/3/library/pprint.html compact impacts the way that sequences are displayed, and a dict is not a sequence.
    So I'm not sure a documentation change is required.

    @iritkatriel iritkatriel added the docs Documentation in the Doc dir label Sep 19, 2020
    @iritkatriel
    Copy link
    Member

    I made PR26967 to break up the paragraph about the constructor params so that it's easier to read, and also added emphasis around the "sequence" point.

    @iritkatriel iritkatriel added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes and removed 3.8 only security fixes labels Jun 30, 2021
    @iritkatriel
    Copy link
    Member

    New changeset 943e77d by Irit Katriel in branch 'main':
    bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… (GH-26967)
    943e77d

    @iritkatriel
    Copy link
    Member

    New changeset 42c2628 by Miss Islington (bot) in branch '3.10':
    bpo-34798: [doc] clearer presentation of pprint.PrettyPrinter constru… (GH-26967) (GH-26990)
    42c2628

    @iritkatriel iritkatriel removed the 3.9 only security fixes label Jul 2, 2021
    @iansedano
    Copy link
    Mannequin

    iansedano mannequin commented Oct 25, 2021

    I came across this and was confused by it too. I also don't understand the justification with not having dicts to be affected by the compact parameter.

    If the "compact form" is having separate entries or elements on one line, instead of having each element separated by a new line, then it seems like inconsistent behavior.

    **If a dict is short enough, it will appear in "compact form", just like a list.**

    If a dict is too long for the width, then each item will appear in "expanded form", also like a list.
    However, the actual compact parameter only affects sequence items. Why is this?

    There is no reason given in bpo-19132. It does mention a review, but it doesn't seem to be available, or I don't know how to get to it, to understand the reason for that decision.

    @posita
    Copy link
    Mannequin

    posita mannequin commented Dec 11, 2021

    Please consider highlighting that dicts are not included in the documentation. While *technically* true, this ...

    compact impacts the way that long sequences (lists, tuples, sets, etc) are formatted. If compact is false (the default) then each item of a sequence will be formatted on a separate line. If compact is true, as many items as will fit within the width will be formatted on each output line.

    ... has several problems.

    First, sequence is a term of art that also has a common meaning. This creates a potential ambiguity in the understanding of the reader. Resolving that ambiguity in this context requires that readers have already internalized that dicts are not Python sequences. Those new to Python may not understand the (to them, subtle) differences between Python's iterables and sequences. Second, the "etc" only expands that ambiguity and invites confusion. Third, the term "items" is strongly associated with dicts and is used several times throughout the paragraph.

    This ...

    According to https://docs.python.org/3/library/pprint.html compact impacts the way that sequences are displayed, and a dict is not a sequence.

    ... is unhelpfully pedantic, and ignorant of the needs of the newcomer (a key demographic of documentation). Documentation is a core product surface with a diverse audience. Rather than focus on technical correctness, documentation authors should focus on accessibility. Redundancy is a feature, not a bug. You can't predict how your reader got to that slice of the documentation. Imagine this as an opportunity to educate or reinforce concepts for readers, not as an opportunity to argue from a position of technical superiority.

    The fact that there are now four developers who have taken their time to file patches, bugs, and comments is pretty strong signal that confusion exists among the audience and that the documentation is insufficient.

    @iritkatriel
    Copy link
    Member

    Don't discuss on a closed issue. Create a new one if there is still a problem.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes 3.11 only security fixes docs Documentation in the Doc dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants