classification
Title: Frame annotation
Type: enhancement Stage:
Components: Interpreter Core Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: doerwalter, ncoghlan, pitrou, vstinner
Priority: normal Keywords: patch

Created on 2013-11-14 17:09 by doerwalter, last changed 2014-01-16 04:29 by ncoghlan.

Files
File name Uploaded Description Edit
frame-annotation.diff doerwalter, 2013-11-14 17:09 review
code-annotation.diff doerwalter, 2013-11-25 12:02 review
Repositories containing patches
http://hg.livinglogic.de/Python-frame-annotation/
Messages (10)
msg202862 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2013-11-14 17:09
This patch adds frame annotations, i.e. it adds an attribute f_annotation to frame objects, a decorator to set this attribute on exceptions and extensions to the traceback machinery that display the annotation in the traceback.
msg202864 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-11-14 17:30
What is the use case of frame annotations?
msg202865 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2013-11-14 17:43
See http://bugs.python.org/issue18861 and the discussion started here: https://mail.python.org/pipermail/python-dev/2013-November/130155.html.

Basically it allows to add context information to a traceback without changing the type of the exception.

In the following example:

   import itertools
   ', '.join(itertools.chain((str(i) for i in range(100)), [42]))

the join method itself adds context information to the TypeError:

   Traceback (most recent call last):
     File "hurz.py", line 2, in <module>
       ', '.join(itertools.chain((str(i) for i in range(100)), [42]))
   TypeError: sequence item 100: expected str instance, int found

i.e. the "sequence item 100" is context information.

However when the exception occurs higher up in the call chain, no such context information is added:

   import itertools

   def foo(x):
      return str(x+1)

   ', '.join(foo(x) for x in itertools.chain(range(100), [None]))

This gives:

   Traceback (most recent call last):
     File "hurz.py", line 6, in <module>
       ', '.join(foo(x) for x in itertools.chain(range(100), [None]))
     File "hurz.py", line 6, in <genexpr>
       ', '.join(foo(x) for x in itertools.chain(range(100), [None]))
     File "hurz.py", line 4, in foo
       return str(x+1)
  TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

With frame annotations the traceback might look like this:

   Traceback (most recent call last):
     File "hurz.py", line 6, in <module>
       ', '.join(foo(x) for x in itertools.chain(range(100), [None]))
     File "hurz.py", line 6, in <genexpr>: sequence item 100
       ', '.join(foo(x) for x in itertools.chain(range(100), [None]))
     File "hurz.py", line 4, in foo
       return str(x+1)
  TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
msg202902 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-11-14 21:56
I think this would need a PEP.
msg202908 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-14 22:37
Aye, it would. The improved exception chaining and frame hiding ideas in
issue 18861 are also related.

The part I particularly like in Walter's suggestion is the idea of a "frame
description" that is displayed in the traceback for that frame, without the
traceback module needing to know the details of other annotations.
msg204329 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2013-11-25 12:02
Here is a new version of the patch. The annotation is done on the code object instead of on the frame object. This avoids two problems: There is no runtime overhead, as the decorator returns the original function and no additional frames show up in the traceback. Since the variables are only known at runtime, the annotation is now a function that does the formatting of the annotation message and gets passed the frame object. With this there is no runtime overhead when no exception is raised and even if an exception is raise, but the traceback is never formatted.
msg204337 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-25 12:52
Code objects are shared amongst multiple function objects, so you
can't store mutable state on them.
msg204364 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2013-11-25 17:12
Do you have an example where code objects are shared? We could attach the annotation formatter to the function object, but unfortunately the function object is now accessible in the traceback.

Note the co_annotation is not the annotation string, rather it is a function that does the formatting of the annotation.
msg204410 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-25 21:19
A nested function will always share the code object between its instances -
the nested code object is a compile time constant stored as part of the
containing code object.

Any code object attribute must be consistent for the lifetime of the object.
msg208233 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-01-16 04:29
Another use case from frame annotations: allowing greenlets and threads to indicate which greenlet or thread a traceback is from whenever it is printed (this is similar to the way the runtime indicates which thread a traceback occurred in when it terminates the the thread, but would apply whenever the uppermost frame is displayed in a thread that isn't the main thread, or in the case of greenlets, the uppermost frame in the greenlet, always)
History
Date User Action Args
2014-01-16 04:29:39ncoghlansetmessages: + msg208233
2013-11-25 21:19:35ncoghlansetmessages: + msg204410
2013-11-25 17:12:38doerwaltersetmessages: + msg204364
2013-11-25 12:52:52ncoghlansetmessages: + msg204337
2013-11-25 12:02:11doerwaltersetfiles: + code-annotation.diff

messages: + msg204329
2013-11-14 22:37:25ncoghlansetmessages: + msg202908
2013-11-14 21:56:56pitrousetnosy: + pitrou, ncoghlan
messages: + msg202902
2013-11-14 17:43:31doerwaltersetmessages: + msg202865
2013-11-14 17:30:05vstinnersetnosy: + vstinner
messages: + msg202864
2013-11-14 17:09:30doerwaltercreate