Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pprint.pprint raises TypeError on dictionaries with user-defined types as keys #54226

Closed
arno mannequin opened this issue Oct 2, 2010 · 8 comments
Closed

pprint.pprint raises TypeError on dictionaries with user-defined types as keys #54226

arno mannequin opened this issue Oct 2, 2010 · 8 comments
Labels
easy stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@arno
Copy link
Mannequin

arno mannequin commented Oct 2, 2010

BPO 10017
Nosy @rhettinger, @amauryfa, @giampaolo, @florentx, @swarmer
Superseder
  • bpo-14998: pprint._safe_key is not always safe enough
  • Files
  • pprint_fix.patch: Fix sorting of class objects and add tests for the issue
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2012-07-21.09:27:04.836>
    created_at = <Date 2010-10-02.20:35:18.179>
    labels = ['easy', 'type-bug', 'library']
    title = 'pprint.pprint raises TypeError on dictionaries with user-defined types as keys'
    updated_at = <Date 2012-07-21.09:27:04.835>
    user = 'https://bugs.python.org/arno'

    bugs.python.org fields:

    activity = <Date 2012-07-21.09:27:04.835>
    actor = 'flox'
    assignee = 'none'
    closed = True
    closed_date = <Date 2012-07-21.09:27:04.836>
    closer = 'flox'
    components = ['Library (Lib)']
    creation = <Date 2010-10-02.20:35:18.179>
    creator = 'arno'
    dependencies = []
    files = ['26433']
    hgrepos = []
    issue_num = 10017
    keywords = ['patch', 'easy']
    message_count = 8.0
    messages = ['117895', '118209', '118214', '118227', '165790', '165817', '165999', '166000']
    nosy_count = 8.0
    nosy_names = ['rhettinger', 'amaury.forgeotdarc', 'giampaolo.rodola', 'rbp', 'arno', 'flox', 'python-dev', 'anton.barkovsky']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = '14998'
    type = 'behavior'
    url = 'https://bugs.python.org/issue10017'
    versions = ['Python 3.2', 'Python 3.3']

    @arno
    Copy link
    Mannequin Author

    arno mannequin commented Oct 2, 2010

    The pprint function in the python 3.1 pprint module fails when
    printing a dictionary containing more than one item and with one item
    being a user-defined type. It seems pprint tries to sort the keys but
    fails, (maybe because calling __lt__ on a user-defined type doesn't
    bind its first argument to 'self'? Looking into pprint.py would
    probably yield the answer).

    This seems related to bpo-3976 but it was fixed in r76389 and
    r76390. My example below fails in r79147. I'm not in a position to check more recent revisions right now.

    In Python 2.6, the following works fine:

    Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) 
    [GCC 4.4.3] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from pprint import pprint
    >>> class A(object): pass
    ... 
    >>> pprint({A:1, 1:2})
    {1: 2, <class __main__.A at 0xb77dc47c>: 1}

    But in Python 3.1, it fails with the error below:

    Python 3.1.2 (r312:79147, Apr 15 2010, 12:35:07) 
    [GCC 4.4.3] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from pprint import pprint
    >>> class A: pass
    ... 
    >>> pprint({A:1, 1:2})
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.1/pprint.py", line 55, in pprint
        printer.pprint(object)
      File "/usr/lib/python3.1/pprint.py", line 132, in pprint
        self._format(object, self._stream, 0, 0, {}, 0)
      File "/usr/lib/python3.1/pprint.py", line 155, in _format
        rep = self._repr(object, context, level - 1)
      File "/usr/lib/python3.1/pprint.py", line 242, in _repr
        self._depth, level)
      File "/usr/lib/python3.1/pprint.py", line 254, in format
        return _safe_repr(object, context, maxlevels, level)
      File "/usr/lib/python3.1/pprint.py", line 296, in _safe_repr
        items = sorted(object.items(), key=_safe_tuple)
      File "/usr/lib/python3.1/pprint.py", line 89, in __lt__
        rv = self.obj.__lt__(other.obj)
    TypeError: expected 1 arguments, got 0

    @arno arno mannequin added the stdlib Python modules in the Lib dir label Oct 2, 2010
    @rbp
    Copy link
    Mannequin

    rbp mannequin commented Oct 8, 2010

    FWIW, the problem still occurs on the most recent release31-maint checkout (as of r85323), and does not happen on py3k (3.2a2).

    @rbp
    Copy link
    Mannequin

    rbp mannequin commented Oct 8, 2010

    If I'm understanding this correctly, this fails on 3.1 and not (although, actually, it does) on py3k/3.2 because:

    • pprint._safe_key.__lt__ checks "rv = self.obj.__lt__(other.obj)" and falls back to id comparison if rv is NotImplemented

    • If the object passed to _safe_key is a class, self.obj.__lt__ will expect *self* as well as the other object. Therefore the verification above fails with "TypeError: expected 1 arguments, got 0". You can see that pprint works with an instance:

    >>> pprint.pprint({A(): 1, 1: 2})
    {<__main__.A object at 0x8594d4c>: 1, 1: 2}
    • Finally, this works on py3k *for your example* because, for some reason, on py3k the comparison is being based on the 1 key. That is, the comparison on _safe_key.__lt__ happens to be 1.__lt__(A), instead of A.__lt__(1). Perhaps hashing changed after the 3.1 release?

    Anyway, py3k still fails when you force the comparison to happen on the class:

    >>> class B(object): pass
    ... 
    >>> pprint.pprint({A: 1, B: 2})
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/rbp/python/dev/py3k/Lib/pprint.py", line 55, in pprint
        printer.pprint(object)
      File "/home/rbp/python/dev/py3k/Lib/pprint.py", line 132, in pprint
        self._format(object, self._stream, 0, 0, {}, 0)
      File "/home/rbp/python/dev/py3k/Lib/pprint.py", line 155, in _format
        rep = self._repr(object, context, level - 1)
      File "/home/rbp/python/dev/py3k/Lib/pprint.py", line 245, in _repr
        self._depth, level)
      File "/home/rbp/python/dev/py3k/Lib/pprint.py", line 257, in format
        return _safe_repr(object, context, maxlevels, level)
      File "/home/rbp/python/dev/py3k/Lib/pprint.py", line 299, in _safe_repr
        items = sorted(object.items(), key=_safe_tuple)
      File "/home/rbp/python/dev/py3k/Lib/pprint.py", line 89, in __lt__
        rv = self.obj.__lt__(other.obj)
    TypeError: expected 1 arguments, got 0
    >>> 

    So, basically, the fix on bpo-3976 does't (always) work when there are classes as dict keys. I think switching from

    rv = self.obj.__lt__(other.obj)
    if rv is NotImplemented:

    to something like

    try:
    rv = self.obj < other.obj
    except TypeError:
    rv = (str(type(self.obj)), id(self.obj)) < \
    (str(type(other.obj)), id(other.obj))

    solves this. Or we can check first whether self.obj is a 'type', but I think it gets convoluted.

    If anyone can confirm that that's the way to go, I can produce one (though it's a trivial one). Raymond?

    @amauryfa
    Copy link
    Member

    amauryfa commented Oct 8, 2010

    A simpler change would to replace:
    rv = self.obj.__lt__(other.obj)
    with
    rv = type(self.obj).__lt__(self.obj, other.obj)

    @florentx
    Copy link
    Mannequin

    florentx mannequin commented Jul 18, 2012

    Confirmed on python 3.2 and 3.3.

    @florentx florentx mannequin added easy type-bug An unexpected behavior, bug, or error labels Jul 18, 2012
    @swarmer
    Copy link
    Mannequin

    swarmer mannequin commented Jul 18, 2012

    Here's a patch with fix and tests.

    Note that class objects are not comparable and have the same type so they
    fall back on sorting by id. Should we sort them by name as a special case instead?

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jul 21, 2012

    New changeset 03cda5360dc6 by Florent Xicluna in branch '3.2':
    Issues bpo-10017 and bpo-14998: Fix TypeError using pprint on dictionaries with unorderable key.
    http://hg.python.org/cpython/rev/03cda5360dc6

    New changeset 4d0dcfbdf45b by Florent Xicluna in branch 'default':
    Issues bpo-10017 and bpo-14998: Fix TypeError using pprint on dictionaries with unorderable key.
    http://hg.python.org/cpython/rev/4d0dcfbdf45b

    @florentx
    Copy link
    Mannequin

    florentx mannequin commented Jul 21, 2012

    Thanks for this patch.
    I've reviewed the issue and merged the patch of issue bpo-14998 which fixes both cases.

    @florentx florentx mannequin closed this as completed Jul 21, 2012
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    easy stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant