classification
Title: Language Reference: Clarify behaviour of yield when generator is not resumed
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: benjamin.peterson, docs@python, eli.bendersky, eric.araujo, ethan.furman, ezio.melotti, georg.brandl, nikratio, pje, python-dev, tim.peters
Priority: normal Keywords: patch

Created on 2011-08-06 18:12 by nikratio, last changed 2014-01-27 03:59 by benjamin.peterson. This issue is now closed.

Files
File name Uploaded Description Edit
patch_v1.diff nikratio, 2011-12-12 02:37 review
yield-docpatch.diff nikratio, 2013-09-08 21:24 Revised patch review
yield-docpatch-rev3.diff nikratio, 2014-01-21 04:11 review
Messages (15)
msg141724 - (view) Author: Nikolaus Rath (nikratio) * Date: 2011-08-06 18:12
From http://docs.python.org/reference/simple_stmts.html#the-yield-statement:

"As of Python version 2.5, the yield statement is now allowed in the try clause of a try ... finally construct. If the generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), the generator-iterator’s close() method will be called, allowing any pending finally clauses to execute."

This strongly suggests that the last-executed yield statement will raise an exception if the generator is closed. If this is the case, it would be great if the documentation could be extended to say what exception is raised.

If this is not the case, it would be great if whatever magic is happening could be documented as well.
msg141726 - (view) Author: Nikolaus Rath (nikratio) * Date: 2011-08-06 18:29
From http://www.python.org/dev/peps/pep-0342/ I believe that the last yield will raise GeneratorExit. So my suggestion is to replace the above mentioned paragraph with:

"""
As of Python version 2.5, yield is an expression rather than a statement and allowed in the try clause of a try ... finally construct. If the generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), the generator-iterator’s close() method will be called, and the yield expression in the generator function will raise GeneratorExit. 

If the generator function raises GeneratorExit (either directly or by not catching it), future calls to the next() method of the generator iterator will raise StopIteration. GeneratorExit exceptions raised by the generator function are catched internally and do not result in a call to sys.excepthook.
"""
msg149270 - (view) Author: Nikolaus Rath (nikratio) * Date: 2011-12-12 02:37
Hmm. Does the total lack responses mean agreement, disagreement or lack of interest? I'm attaching a patch against Python 3.3 in the hope of moving this forward.
msg196884 - (view) Author: Nikolaus Rath (nikratio) * Date: 2013-09-04 03:00
*ping*
msg196907 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2013-09-04 13:03
Why guess... did you try it in the code?

Trying has another goal - it would be nice to have a short code sample here demonstrating what's happening. The paragraph you're quoting seems obscure to me, with or without the fix.
msg196972 - (view) Author: Nikolaus Rath (nikratio) * Date: 2013-09-05 00:27
On 09/04/2013 06:03 AM, Eli Bendersky wrote:
> Why guess... did you try it in the code?

I don't follow... why guess what? And try what in code?

> it would be nice to have a short code sample here demonstrating what's happening. The paragraph you're quoting seems obscure to me, with or without the fix.

Sounds reasonable. I'll revise the patch and add an example.

Best,

   -Nikolaus

-- 
 »Time flies like an arrow, fruit flies like a Banana.«

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C
msg196976 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2013-09-05 03:48
I think the docs are already clear:  they say "the generator-iterator’s close() method will be called".  That's all that needs to be said:  now go look at the docs for generator.close().  They explain _all_ that close() does, and it would be a Bad Idea to duplicate all that in this part of the docs too.  (And, yes, for a start, close() "Raises a GeneratorExit at the point where the generator function was paused".)

It would certainly help if the "close" (in "the generator-iterator’s close() method will be called") were an active hyperlink in the docs.
msg197009 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2013-09-05 15:57
On Wed, Sep 4, 2013 at 5:27 PM, Nikolaus Rath <report@bugs.python.org>wrote:

>
> Nikolaus Rath added the comment:
>
> On 09/04/2013 06:03 AM, Eli Bendersky wrote:
> > Why guess... did you try it in the code?
>
> I don't follow... why guess what? And try what in code?
>

I was referring to this part of the original report:

"""
This strongly suggests that the last-executed yield statement will raise an
exception if the generator is closed. If this is the case, it would be
great if the documentation could be extended to say what exception is
raised.

If this is not the case, it would be great if whatever magic is happening
could be documented as well.
"""

If this is the case, if this is not the case... I was just saying that you
can provide some sample code in the issue that demonstrates what *actually*
happens and whether it's not explained well enough in the documentation.
msg197323 - (view) Author: Nikolaus Rath (nikratio) * Date: 2013-09-08 21:24
I've attached an updated patch.

I agree with Tim that the docs on the yield expression are already very good, so instead of extending the yield statement documentation I have shortened it to instead reduce the overlap that already exists.

In the yield expression documentation, I have replaced :keyword:`yield` with an un-tagged "yield" (because the former results in a link to the yield statement) and added '.. or yield statement' to the parts that apply to both yield expressions and statements.

I have also added hyperlinks to the relevant descriptions of the generator methods.
msg207667 - (view) Author: Nikolaus Rath (nikratio) * Date: 2014-01-08 05:13
*ping*

Any comments on the updated patch? Can it be applied?
msg208520 - (view) Author: Nikolaus Rath (nikratio) * Date: 2014-01-20 04:43
(adding the documentation experts from http://docs.python.org/devguide/experts.html to noisy in the hope to push this forward)
msg208527 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-01-20 07:45
Hmm, the yield statement is now basically the same as any expression statement -- just like a function call.  The only reason it's a separate grammar symbol is that otherwise yield expressions would have to be parenthesized.

Can the patch be updated to make this a bit clearer?  Otherwise I like the approach of moving the info to the expressions page.
msg208609 - (view) Author: Nikolaus Rath (nikratio) * Date: 2014-01-21 04:11
That makes sense. Attached is an updated patch. It removes most of the duplication, and clearly says that there is no semantic difference between the yield statement and the yield expression at all.

I also moved the "see also" block to follow the description of the yield expression. At least for me, it was rather easy to miss in its current position (at the end of the second example subsection). The rest of the code churn in that section is just removing all the (presumably accidental) links to the yield statement when referring to a yield expression.
msg209388 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-01-27 03:58
New changeset e02da391741f by Benjamin Peterson in branch '3.3':
eliminate redundancy between yield stmt and yield expr docs (closes #12704)
http://hg.python.org/cpython/rev/e02da391741f

New changeset c6b4a5354c23 by Benjamin Peterson in branch 'default':
merge 3.3 (closes #12704)
http://hg.python.org/cpython/rev/c6b4a5354c23
msg209389 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-01-27 03:59
Thank you. In the future, please remember to wrap paragraphs to 80 chars and run "make patchcheck" to cleanup whitespace issues.
History
Date User Action Args
2014-01-27 03:59:48benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg209389
2014-01-27 03:58:51python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg209388

resolution: fixed
stage: resolved
2014-01-21 04:11:40nikratiosetfiles: + yield-docpatch-rev3.diff

messages: + msg208609
2014-01-20 07:45:30georg.brandlsetmessages: + msg208527
2014-01-20 04:43:04nikratiosetnosy: + georg.brandl, ezio.melotti, eric.araujo
messages: + msg208520
2014-01-18 02:46:15ethan.furmansetnosy: + ethan.furman
2014-01-08 05:13:54nikratiosetmessages: + msg207667
2013-09-08 21:24:44nikratiosetfiles: + yield-docpatch.diff

messages: + msg197323
2013-09-05 15:57:57eli.benderskysetmessages: + msg197009
2013-09-05 03:48:15tim.peterssetnosy: + tim.peters
messages: + msg196976
2013-09-05 00:27:15nikratiosetmessages: + msg196972
2013-09-04 13:03:48eli.benderskysetnosy: + eli.bendersky
messages: + msg196907
2013-09-04 07:31:41berker.peksagsetnosy: + pje

versions: - Python 2.6, Python 3.1, Python 3.2
2013-09-04 03:00:22nikratiosetmessages: + msg196884
2011-12-12 02:37:43nikratiosettitle: Language References does not specify exception raised by final yield() -> Language Reference: Clarify behaviour of yield when generator is not resumed
2011-12-12 02:37:07nikratiosetfiles: + patch_v1.diff
keywords: + patch
messages: + msg149270
2011-09-11 13:40:29nikratiosetversions: + Python 3.1, Python 3.2, Python 3.3, Python 3.4
2011-08-06 18:29:09nikratiosetmessages: + msg141726
2011-08-06 18:12:26nikratiocreate