Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pprint handling of dict subclasses that override __repr__ #84175

Closed
palakjha mannequin opened this issue Mar 17, 2020 · 17 comments
Closed

pprint handling of dict subclasses that override __repr__ #84175

palakjha mannequin opened this issue Mar 17, 2020 · 17 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@palakjha
Copy link
Mannequin

palakjha mannequin commented Mar 17, 2020

BPO 39994
Nosy @freddrake, @rhettinger, @ericvsmith, @serhiy-storchaka, @Mariatta, @Palakjha, @iritkatriel
PRs
  • bpo-39994: Removed redundant code. #19046
  • bpo-39994: Fix pprint handling of dict subclasses that override __repr__ #21892
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-08-30.19:46:24.371>
    created_at = <Date 2020-03-17.16:19:33.505>
    labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
    title = 'pprint handling of dict subclasses that override __repr__'
    updated_at = <Date 2020-08-30.19:46:24.371>
    user = 'https://github.com/palakjha'

    bugs.python.org fields:

    activity = <Date 2020-08-30.19:46:24.371>
    actor = 'eric.smith'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-08-30.19:46:24.371>
    closer = 'eric.smith'
    components = ['Library (Lib)']
    creation = <Date 2020-03-17.16:19:33.505>
    creator = 'palakjha'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 39994
    keywords = ['patch']
    message_count = 17.0
    messages = ['364442', '375261', '375270', '375271', '375279', '375280', '375425', '375451', '375453', '375465', '375467', '375470', '375473', '375485', '375491', '376117', '376120']
    nosy_count = 7.0
    nosy_names = ['fdrake', 'rhettinger', 'eric.smith', 'serhiy.storchaka', 'Mariatta', 'palakjha', 'iritkatriel']
    pr_nums = ['19046', '21892']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue39994'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    @palakjha
    Copy link
    Mannequin Author

    palakjha mannequin commented Mar 17, 2020

    In the PrettyPrinter._format method, since self._dispatch has dict.__repr__ [key] mapped to self._pprint_dict [value] the elif block is not needed. Its work is already being done by the if block above, which searches self._dispatch to fetch the appropriate value.

    @palakjha palakjha mannequin added 3.7 (EOL) end of life type-feature A feature request or enhancement 3.8 only security fixes 3.9 only security fixes and removed 3.7 (EOL) end of life labels Mar 17, 2020
    @iritkatriel
    Copy link
    Member

    This change makes a difference in semantics (perhaps a good one) in this case:

    import pprint
    
    class MyDict(dict):
        def __repr__(self):
            return 'I do my own thing'*50
    
    if __name__ == '__main__':
        d=MyDict()
        for i in range(50):
            d['a%s'%i] = i
    
        pprint.pprint(d)

    Before the change MyDict.__repr__ is ignored and the dict contents are printed. After the change it prints "I do my own thing" 50 times.

    @serhiy-storchaka
    Copy link
    Member

    Yes, this code was kept for backward compatibility when the dispatch mapping was added.

    @freddrake
    Copy link
    Member

    Adding serhiy.storchaka to nosy list since it looks like he introduced the isinstance check on purpose (see bpo-23741).

    Though bpo-23741 asserts that no behavior was changed with the patch applied then, reading through the change leads me to think this did change for cases like the MyDict example from iritkatriel.

    The intent of the original code was that MyDict.__repr__ would be used; only the small handful of "known" reprs would be handled specially.

    @serhiy-storchaka
    Copy link
    Member

    bpo-23741 did not change the behavior. The specified code was added specially to preserve the existing behavior. For other base types the condition was like issubclass(typ, list) and r is list.__repr__, but for dict it was just issubclass(typ, dict).

    Actually the initial behavior was changed by bad3c88 (no issue number), with adding support of OrderedDict.

    @freddrake
    Copy link
    Member

    Ah, good find! It's been so long since the first version of that code, but the implementation was surprisingly nuanced.

    @Mariatta
    Copy link
    Member

    @palak Kumar Jha, you created a PR for this ticket but it seems that your GitHub account is inactive/has been removed. Do you have a new GitHub account? Are you still interested in continuing on this PR?

    @ericvsmith
    Copy link
    Member

    I went ahead and closed the PR. Either @palak Kumar Jha or someone else can create a new PR. The suggestions in the original PR should be addresses.

    @iritkatriel
    Copy link
    Member

    Thanks, I'll create a new PR and credit @palak Kumar Jha as co-author.

    @iritkatriel iritkatriel added the stdlib Python modules in the Lib dir label Aug 15, 2020
    @iritkatriel iritkatriel changed the title Redundant code in pprint module. pprint handling of dict subclasses that override __repr__ Aug 15, 2020
    @iritkatriel iritkatriel added type-bug An unexpected behavior, bug, or error stdlib Python modules in the Lib dir and removed type-feature A feature request or enhancement labels Aug 15, 2020
    @iritkatriel iritkatriel changed the title Redundant code in pprint module. pprint handling of dict subclasses that override __repr__ Aug 15, 2020
    @iritkatriel iritkatriel added type-bug An unexpected behavior, bug, or error and removed type-feature A feature request or enhancement labels Aug 15, 2020
    @iritkatriel
    Copy link
    Member

    While writing the tests I see that it's even more interesting:

    (Pdb) dd = MyDict()
    (Pdb) pprint.pformat(dd)
    '{}'
    (Pdb) pprint.saferepr(dd)
    'I do my own thing ....'

    @freddrake
    Copy link
    Member

    And that is why the original code was checking not only for the type, but the actual __repr__ method itself.

    I think the current behavior is broken.

    @ericvsmith
    Copy link
    Member

    I realize it might break some corner cases, but I really think we should re-write pprint to use functools.singledispatch. Or if the breakage isn't acceptable, abandon it and create a new module that does use singledispatch. That way it would be easily extensible.

    pprint currently works with a dispatch table (_dispatch), it's just hidden.

    @fdrake: From the comments: "If you find it useful, thank small children who sleep at night". I assume they're not small anymore!

    @iritkatriel
    Copy link
    Member

    import pprint
    
    class MyDict(dict):
        def __repr__(self):
            return '*'*len(dict.__repr__(self))
    
    if __name__ == '__main__':
        d=MyDict({})
        print('pprint.pformat(d):\n%s' % pprint.pformat(d))
        print('pprint.pformat(d, width=1, indent=0):\n%s' % pprint.pformat(d, width=1, indent=0))

    Output:

    pprint.pformat(d):
    **
    pprint.pformat(d, width=1, indent=0):
    {}

    @iritkatriel
    Copy link
    Member

    I've created a new PR, with two commits. The first has just tests added, which show the problems and assert that the bug is there. The second has the fix and update to the tests. Let me know what you think.

    @iritkatriel iritkatriel added 3.7 (EOL) end of life 3.10 only security fixes labels Aug 15, 2020
    @ericvsmith
    Copy link
    Member

    Only 3.8 - 3.10 would be eligible for this fix. 3.7 is getting only security fixes.

    @ericvsmith ericvsmith removed 3.7 (EOL) end of life labels Aug 15, 2020
    @serhiy-storchaka
    Copy link
    Member

    New changeset 582f137 by Irit Katriel in branch 'master':
    bpo-39994: Fix pprint handling of dict subclasses that override __repr__ (GH-21892)
    582f137

    @iritkatriel
    Copy link
    Member

    Thank you Serhiy. This ticket can be closed now.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants