classification
Title: Delegating generator is not always visible to debugging tools such as inspect & pdb
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, anacrolix, benjamin.peterson, georg.brandl, ncoghlan, python-dev
Priority: normal Keywords: patch

Created on 2012-03-08 21:11 by Mark.Shannon, last changed 2012-03-16 14:39 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
yieldfrom.patch Mark.Shannon, 2012-03-15 18:42 Patch to revision 52597f888e7a review
yieldfrom.patch Mark.Shannon, 2012-03-15 19:07 review
magic.patch Mark.Shannon, 2012-03-16 14:37 review
Messages (13)
msg155182 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2012-03-08 21:11
Delegating generators do not show always up in stack traces, such as inspect.stack().
The show up during the first delegation, but not thereafter.

I have attached a patch. It alters the way the YIELD_FROM bytecode works; it loops back on itself. This ensures the delegator's frame is always in the trace.

Unfortunately I started working on it before Benjamin fixed issue 14220
(rev 3357eac1ba62). By the nature of it, this patch necessarily fixes most of issue  14220, so I have just included the tests for issue 14220
in this patch as well.
So in order to apply this, 3357eac1ba62 will have to be backed out.
Sorry for the overlap.
msg155380 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2012-03-11 11:00
Mark's patch looks good on a quick read through (although I still need to double check where f_lasti gets incremented to be happy the new comment about YIELD_FROM is accurate). 

If someone gets to this soon, great, otherwise I'll deal with it after I finish moving house.
msg155894 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-03-15 15:31
I would appreciate the patch could be regenerated against the latest default.
msg155932 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2012-03-15 18:42
Updated Patch
msg155936 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-03-15 18:54
Mercurial complains when trying to apply the patch:

applying yieldfrom.patch
patching file Lib/test/test_pep380.py
Hunk #2 succeeded at 921 with fuzz 2 (offset 72 lines).
abort: bad hunk #1 @@ -20,7 +20,6 @@
 (7 7 7 6)
msg155937 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2012-03-15 19:07
Could you try this new patch (with white space changes included)?
msg155955 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-03-15 20:38
New changeset 72556ff86828 by Benjamin Peterson in branch 'default':
perform yield from delegation by repeating YIELD_FROM opcode (closes #14230)
http://hg.python.org/cpython/rev/72556ff86828
msg156031 - (view) Author: Matt Joiner (anacrolix) Date: 2012-03-16 14:13
This changeset has broken something. All I get is a confusing backtrace ending with:

  File "/home/matt/src/python-torrent/torrent/bencode.py", line 15, in encode
    yield from encode(k)
AttributeError: 'list_iterator' object has no attribute 'send'

Backing out the changeset, and there is no problem. Let me know if there's some way to provide more information.
msg156032 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-03-16 14:15
2012/3/16 Matt Joiner <report@bugs.python.org>:
>
> Matt Joiner <anacrolix@gmail.com> added the comment:
>
> This changeset has broken something. All I get is a confusing backtrace ending with:
>
>  File "/home/matt/src/python-torrent/torrent/bencode.py", line 15, in encode
>    yield from encode(k)
> AttributeError: 'list_iterator' object has no attribute 'send'
>
> Backing out the changeset, and there is no problem. Let me know if there's some way to provide more information.

Try removing all the pycs.
msg156033 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-03-16 14:15
Nevermind, I think I see the problem.

2012/3/16 Benjamin Peterson <report@bugs.python.org>:
>
> Benjamin Peterson <benjamin@python.org> added the comment:
>
> 2012/3/16 Matt Joiner <report@bugs.python.org>:
>>
>> Matt Joiner <anacrolix@gmail.com> added the comment:
>>
>> This changeset has broken something. All I get is a confusing backtrace ending with:
>>
>>  File "/home/matt/src/python-torrent/torrent/bencode.py", line 15, in encode
>>    yield from encode(k)
>> AttributeError: 'list_iterator' object has no attribute 'send'
>>
>> Backing out the changeset, and there is no problem. Let me know if there's some way to provide more information.
>
> Try removing all the pycs.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue14230>
> _______________________________________
msg156034 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-03-16 14:22
Can you possibly create a small example that fails?
msg156038 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2012-03-16 14:37
The magic number has not been updated

Patch attached.
msg156040 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-03-16 14:39
New changeset c0a6569fdad6 by Benjamin Peterson in branch 'default':
in 72556ff86828, I should have updated the magic as well as the comment (#14230)
http://hg.python.org/cpython/rev/c0a6569fdad6
History
Date User Action Args
2012-03-16 14:39:29python-devsetmessages: + msg156040
2012-03-16 14:37:08Mark.Shannonsetfiles: + magic.patch

messages: + msg156038
2012-03-16 14:22:31benjamin.petersonsetmessages: + msg156034
2012-03-16 14:15:57benjamin.petersonsetmessages: + msg156033
2012-03-16 14:15:14benjamin.petersonsetmessages: + msg156032
2012-03-16 14:13:56anacrolixsetnosy: + anacrolix
messages: + msg156031
2012-03-15 20:38:28python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg155955

resolution: fixed
stage: resolved
2012-03-15 19:07:01Mark.Shannonsetfiles: + yieldfrom.patch

messages: + msg155937
2012-03-15 18:54:45benjamin.petersonsetmessages: + msg155936
2012-03-15 18:43:28Mark.Shannonsetfiles: - yieldfrom.patch
2012-03-15 18:42:41Mark.Shannonsetfiles: + yieldfrom.patch

messages: + msg155932
2012-03-15 15:31:06benjamin.petersonsetmessages: + msg155894
2012-03-11 11:00:10ncoghlansetmessages: + msg155380
2012-03-11 10:16:23eric.araujosetnosy: + georg.brandl, ncoghlan, benjamin.peterson
2012-03-08 21:11:50Mark.Shannoncreate