classification
Title: Incorrect stack traces when re-entering a generator/coroutine stack via .throw()
Type: Stage: patch review
Components: Interpreter Core Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, chris.jerdonek, miss-islington, njs
Priority: normal Keywords: patch

Created on 2017-02-17 14:23 by njs, last changed 2020-09-04 23:07 by miss-islington.

Files
File name Uploaded Description Edit
weird-throw-stack.py njs, 2017-02-17 14:23
Pull Requests
URL Status Linked Edit
PR 19896 merged chris.jerdonek, 2020-05-04 06:54
PR 21416 closed miss-islington, 2020-07-09 13:27
PR 22106 merged miss-islington, 2020-09-04 22:35
Messages (5)
msg288010 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-02-17 14:23
The attached script sets up a stack of generators calling each other via 'yield from': f yields from g yield from h. Then the generator at the bottom of the stack yields.

Before they yield, sys._getframe shows the expected stack (or if you put in traceback.print_stack() you get the same thing).

After they yield, it depends: if the generator is resumed via 'gen.send(None)', then the stack looks sensible. But if the generator is resumed via 'gen.throw(...)', then the stack is weird: Objects/genobject.c:_gen_throw implements 'yield from' in an weird manual way, where it first directly resumes the innermost generator frame, and then propagates any result to the next generator frame, etc. So the output I get from the sample script is:

~$ python3.6 weird-throw-stack.py
-- f before yielding --
f
<module>
-- g before yielding --
g
f
<module>
-- h before yielding --
h
g
f
<module>
-- h after yielding --
h
<module>
-- g after yielding --
g
<module>
-- f after yielding --
f
<module>

This causes problems for stack-based profiling (https://github.com/vmprof/vmprof-python/issues/117), debuggers, and other tools that need to introspect the stack.
msg368012 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2020-05-04 07:16
I proposed a PR to fix this: https://github.com/python/cpython/pull/19896
msg369417 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2020-05-20 04:49
I just filed a related issue to this that's also similar to issue 29587:
https://bugs.python.org/issue40694
msg373404 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2020-07-09 13:27
New changeset 8b33961e4bc4020d8b2d5b949ad9d5c669300e89 by Chris Jerdonek in branch 'master':
bpo-29590: fix stack trace for gen.throw() with yield from (#19896)
https://github.com/python/cpython/commit/8b33961e4bc4020d8b2d5b949ad9d5c669300e89
msg376414 - (view) Author: miss-islington (miss-islington) Date: 2020-09-04 23:07
New changeset e92219d8f864a1a8eb381d98d5df4f1aa767dacb by Miss Islington (bot) in branch '3.9':
bpo-29590: fix stack trace for gen.throw() with yield from (GH-19896)
https://github.com/python/cpython/commit/e92219d8f864a1a8eb381d98d5df4f1aa767dacb
History
Date User Action Args
2020-09-04 23:07:22miss-islingtonsetmessages: + msg376414
2020-09-04 22:35:23miss-islingtonsetpull_requests: + pull_request21191
2020-07-09 13:27:45miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request20565
2020-07-09 13:27:31Mark.Shannonsetnosy: + Mark.Shannon
messages: + msg373404
2020-05-20 04:49:18chris.jerdoneksetmessages: + msg369417
2020-05-04 07:16:07chris.jerdoneksetmessages: + msg368012
2020-05-04 06:54:34chris.jerdoneksetkeywords: + patch
stage: patch review
pull_requests: + pull_request19207
2020-05-03 10:26:15chris.jerdoneksetversions: + Python 3.8, Python 3.9
2020-05-02 11:29:03chris.jerdoneksetnosy: + chris.jerdonek
2017-04-11 13:02:48Mariattasetcomponents: + Interpreter Core
2017-02-17 14:23:48njscreate