classification
Title: traceback module can't format/print unhashable exceptions
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Chris Barth, Jelle Zijlstra, Trundle, berker.peksag, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-11-03 21:22 by Trundle, last changed 2017-06-14 20:30 by Chris Barth.

Files
File name Uploaded Description Edit
unhashable_exceptions.diff Trundle, 2016-11-03 21:22 Patch against default branch review
issue28603-list.patch Jelle Zijlstra, 2016-12-13 08:01 review
issue28603-ignore.patch Jelle Zijlstra, 2016-12-13 08:01 review
issue28603-list.patch Jelle Zijlstra, 2016-12-13 08:01 review
Messages (6)
msg280022 - (view) Author: Andreas Stührk (Trundle) * Date: 2016-11-03 21:22
The traceback module tries to handle loops caused by an exception's __cause__ or __context__ attributes when printing tracebacks. To do so, it adds already seen exceptions to a set. Unfortunately, it doesn't handle unhashable exceptions:

>>> class E(Exception): __hash__ = None
...
>>> traceback.print_exception(E, E(), None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/traceback.py", line 100, in print_exception
    type(value), value, tb, limit=limit).format(chain=chain):
  File "/usr/lib/python3.5/traceback.py", line 439, in __init__
    _seen.add(exc_value)
TypeError: unhashable type: 'E'

CPython's internal exception printing pretty much does the same, except it ignores any exception while operating on the seen set (see https://hg.python.org/cpython/file/8ee4ed577c03/Python/pythonrun.c#l813 ff).

Attached is a patch that makes the traceback module ignore TypeErrors while operating on the seen set. It also adds a (minimal) test.
msg280233 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-11-07 20:31
Thanks for the patch. Out of curiosity, how did you get an unhashable exception? Is there a way to reproduce this without creating a custom exception and set __hash__ to None? In other words, what's your use case?
msg280237 - (view) Author: Andreas Stührk (Trundle) * Date: 2016-11-07 20:40
It was reported as bug to a project that uses the traceback module (https://github.com/bpython/bpython/issues/651). Parsley (https://pypi.python.org/pypi/Parsley) is at least one library that uses unhashable exceptions, although I guess it's by accident: it defines __eq__, but not __hash__ and that makes the exception unhashable under Python 3.
msg283081 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * Date: 2016-12-13 08:01
I ran into this bug through Thrift-generated exception classes (also reported there as https://issues.apache.org/jira/browse/THRIFT-4002).

I've added a few potential solutions:
- issue28603-listset.patch turns the seen set into a list if hashing fails. However, this adds a lot of complexity, especially in C, and because seen is changed halfway through the recursion, we may end up showing an exception twice.
- issue28603-list.patch uses a list instead of a set for seen. This is theoretically slower, but in practice it seems unlikely that exception __cause__ and __context__ would nest deep enough for this to be an issue.
- issue28603-ignore.patch takes a similar approach to Trundle's patch and just gives up when the value is not hashable. This means we lose cause/context information for these exceptions.

I prefer issue28603-list.patch.
msg283095 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-13 11:15
There is no issue28603-listset.patch.
msg296034 - (view) Author: Chris Barth (Chris Barth) Date: 2017-06-14 20:30
There are now several patches for this problem, which also affects me. What are the next steps to get this resolved?
History
Date User Action Args
2017-06-14 20:30:43Chris Barthsetmessages: + msg296034
2017-06-14 20:29:04Chris Barthsetnosy: + Chris Barth
2016-12-13 11:15:56serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg283095
2016-12-13 08:01:49Jelle Zijlstrasetfiles: + issue28603-list.patch
2016-12-13 08:01:37Jelle Zijlstrasetfiles: + issue28603-ignore.patch
2016-12-13 08:01:16Jelle Zijlstrasetfiles: + issue28603-list.patch
nosy: + Jelle Zijlstra
messages: + msg283081

2016-11-07 20:40:32Trundlesetmessages: + msg280237
2016-11-07 20:31:27berker.peksagsetversions: + Python 3.5, Python 3.6, Python 3.7
nosy: + berker.peksag

messages: + msg280233

type: behavior
stage: patch review
2016-11-03 21:22:14Trundlecreate