Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

traceback module can't format/print unhashable exceptions #72789

Closed
Trundle mannequin opened this issue Nov 3, 2016 · 11 comments
Closed

traceback module can't format/print unhashable exceptions #72789

Trundle mannequin opened this issue Nov 3, 2016 · 11 comments
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@Trundle
Copy link
Mannequin

Trundle mannequin commented Nov 3, 2016

BPO 28603
Nosy @terryjreedy, @Trundle, @berkerpeksag, @serhiy-storchaka, @JelleZijlstra, @zaneb
PRs
  • bpo-28603: Fix exception chaining for unhashable exceptions #4014
  • [3.6] bpo-28603: Fix formatting tracebacks for unhashable exceptions (GH-4014) #4024
  • Files
  • unhashable_exceptions.diff: Patch against default branch
  • issue28603-list.patch
  • issue28603-ignore.patch
  • issue28603-list.patch
  • hashex.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2017-10-17.22:17:00.634>
    created_at = <Date 2016-11-03.21:22:14.375>
    labels = ['3.7', 'type-bug', 'library']
    title = "traceback module can't format/print unhashable exceptions"
    updated_at = <Date 2017-10-17.22:17:00.632>
    user = 'https://github.com/Trundle'

    bugs.python.org fields:

    activity = <Date 2017-10-17.22:17:00.632>
    actor = 'serhiy.storchaka'
    assignee = 'none'
    closed = True
    closed_date = <Date 2017-10-17.22:17:00.634>
    closer = 'serhiy.storchaka'
    components = ['Library (Lib)']
    creation = <Date 2016-11-03.21:22:14.375>
    creator = 'Trundle'
    dependencies = []
    files = ['45342', '45870', '45871', '45872', '47223']
    hgrepos = []
    issue_num = 28603
    keywords = ['patch']
    message_count = 11.0
    messages = ['280022', '280233', '280237', '283081', '283095', '296034', '304485', '304534', '304535', '304538', '304539']
    nosy_count = 7.0
    nosy_names = ['terry.reedy', 'Trundle', 'berker.peksag', 'serhiy.storchaka', 'JelleZijlstra', 'Chris Barth', 'zaneb']
    pr_nums = ['4014', '4024']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue28603'
    versions = ['Python 3.6', 'Python 3.7']

    @Trundle
    Copy link
    Mannequin Author

    Trundle mannequin commented Nov 3, 2016

    The traceback module tries to handle loops caused by an exception's __cause__ or __context__ attributes when printing tracebacks. To do so, it adds already seen exceptions to a set. Unfortunately, it doesn't handle unhashable exceptions:

    >>> class E(Exception): __hash__ = None
    ...
    >>> traceback.print_exception(E, E(), None)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.5/traceback.py", line 100, in print_exception
        type(value), value, tb, limit=limit).format(chain=chain):
      File "/usr/lib/python3.5/traceback.py", line 439, in __init__
        _seen.add(exc_value)
    TypeError: unhashable type: 'E'

    CPython's internal exception printing pretty much does the same, except it ignores any exception while operating on the seen set (see https://hg.python.org/cpython/file/8ee4ed577c03/Python/pythonrun.c#l813 ff).

    Attached is a patch that makes the traceback module ignore TypeErrors while operating on the seen set. It also adds a (minimal) test.

    @Trundle Trundle mannequin added the stdlib Python modules in the Lib dir label Nov 3, 2016
    @berkerpeksag
    Copy link
    Member

    Thanks for the patch. Out of curiosity, how did you get an unhashable exception? Is there a way to reproduce this without creating a custom exception and set __hash__ to None? In other words, what's your use case?

    @berkerpeksag berkerpeksag added 3.7 (EOL) end of life type-bug An unexpected behavior, bug, or error labels Nov 7, 2016
    @Trundle
    Copy link
    Mannequin Author

    Trundle mannequin commented Nov 7, 2016

    It was reported as bug to a project that uses the traceback module (bpython/bpython#651). Parsley (https://pypi.python.org/pypi/Parsley) is at least one library that uses unhashable exceptions, although I guess it's by accident: it defines __eq__, but not __hash__ and that makes the exception unhashable under Python 3.

    @JelleZijlstra
    Copy link
    Member

    I ran into this bug through Thrift-generated exception classes (also reported there as https://issues.apache.org/jira/browse/THRIFT-4002).

    I've added a few potential solutions:

    • bpo-28603-listset.patch turns the seen set into a list if hashing fails. However, this adds a lot of complexity, especially in C, and because seen is changed halfway through the recursion, we may end up showing an exception twice.
    • bpo-28603-list.patch uses a list instead of a set for seen. This is theoretically slower, but in practice it seems unlikely that exception __cause__ and __context__ would nest deep enough for this to be an issue.
    • bpo-28603-ignore.patch takes a similar approach to Trundle's patch and just gives up when the value is not hashable. This means we lose cause/context information for these exceptions.

    I prefer bpo-28603-list.patch.

    @serhiy-storchaka
    Copy link
    Member

    There is no bpo-28603-listset.patch.

    @ChrisBarth
    Copy link
    Mannequin

    ChrisBarth mannequin commented Jun 14, 2017

    There are now several patches for this problem, which also affects me. What are the next steps to get this resolved?

    @zaneb
    Copy link
    Mannequin

    zaneb mannequin commented Oct 16, 2017

    I also encountered this issue, and I'd like to further note that it is extraordinarily difficult to debug without access to stderr. If your logging facility uses the traceback module then it will be unable to record both the original (unhashable) exception, and the TypeError that it triggers (since the original exception is its __context__). The effect is that execution of a thread appears to simply stop without rhyme or reason (in fact it is still executing, but that fact is not apparent from the logs). I'm attaching a short reproducer that demonstrates this effect.

    I believe the trade-offs inherent in the attached patches (either stopping the chain at an unhashable exception, or using a less-efficient data structure) can be avoided by comparing for identity rather than equality. The purpose of the check is to avoid an infinite loop; whether the objects compare as equal is something that is up to the author of the class, and has no relevance here except insofar as that objects ought to compare equal to themselves.

    I submitted a pull request (bpo-4014) with an implementation of that.

    @terryjreedy
    Copy link
    Member

    As I understand it, the patch amounts to ignoring any custom __eq__ and __hash__ on an Exception class when printing tracebacks and, in effect, using the default id-based versions inherited from object, as is being assumed. This seems right to me in this context.

    @serhiy-storchaka
    Copy link
    Member

    New changeset de86073 by Serhiy Storchaka (Zane Bitter) in branch 'master':
    bpo-28603: Fix formatting tracebacks for unhashable exceptions (bpo-4014)
    de86073

    @serhiy-storchaka
    Copy link
    Member

    New changeset 2712247 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
    [3.6] bpo-28603: Fix formatting tracebacks for unhashable exceptions (GH-4014) (bpo-4024)
    2712247

    @serhiy-storchaka
    Copy link
    Member

    Thank you for your contribution Zane.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants