Title: Implement generator interface in itertools.chain.
Type: enhancement Stage: resolved
Components: Extension Modules Versions: Python 3.4
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, ncoghlan, pyos, rhettinger, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2012-10-06 14:52 by pyos, last changed 2012-10-09 03:08 by terry.reedy. This issue is now closed.

File name Uploaded Description Edit
itertools-chain-doc.diff pyos, 2012-10-06 15:00 Change `for element in it: yield element` to `yield from it` in the documentation. review
itertools-chain-send-throw-and-close-2.diff pyos, 2012-10-07 07:48 Add `send`, `throw`, and `close` methods to `itertools.chain` objects. review
Messages (7)
msg172204 - (view) Author: (pyos) Date: 2012-10-06 14:52
Since "yield from" made it into Python 3.3, I think it would be useful to chain multiple generators and still get a generator, not an iterator. That is, the following code:

def f():
    yield from itertools.chain(A, B, C)

should be (at least roughly) equivalent to

def f():
    yield from A
    yield from B
    yield from C

while still allowing to send() values to whichever subgenerator is currently running or throw() exceptions inside them.

The attached patch adds this functionality to itertools.chain objects.
msg172280 - (view) Author: (pyos) Date: 2012-10-07 07:48
Updated the patch. Thanks Serhiy Storchaka for comments on the previous one.
msg172422 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2012-10-08 22:42
TBH I don't think that using "yield from" in the docs makes things clearer at all -- it just requires readers to understand a rather esoteric feature.
msg172423 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2012-10-08 22:43
I also don't think that this is a desirable feature.
msg172436 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2012-10-09 02:48
I am going to reject this unless at least one other core developer supports it.  Extensive discussion on python-ideas has found very few supporters besides Serhiy Storchaka.
msg172437 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2012-10-09 03:01
As discussed on python-ideas: the iterator interface is narrower than the generator interface. Tools for working with iterators are *expected* to lose the generator specific details, just as for loops do, especially when they deal with multiple iterators.

When Greg Ewing's PEP 3152 about cofunctions was discussed in the past, the possibility of a C level API to allow other objects to behave like generators was brought up - if such an API is ever adopted, then it *may* make sense to use it in itertools.

In the meantime, it's simpler if itertools is consistent about ignoring the .send(), .throw() and return value semantics of generators-as-coroutines.
msg172438 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-10-09 03:08
In the python-ideas thread, "Propagating StopIteration value",
Guido further elaborated "But that just seems to perpetuate the idea that you have, which IMO is wrongheaded. Itertools is for iterators, and all the extra generator features make no sense for it." I agree with this completely.

Given Raymond's consistent rejection of attempts to complexify itertools, I am 90% sure he would reject this also. (If he disagrees, it is easy to re-open.)

You and Serhiy seem to miss that the simplicity of the iterator interface as a lowest common denominator is a feature. It works for its intended purpose. It was fixed in 2.2 and so far not subject to change. And it can effectively be mixed in to other classes to produce iterator + whatever classes. Files are one example -- they are iterators, context managers, and io method objects. Generators, with its additions, are another example of augmented iterator.

"I think it would be useful to chain multiple generators and still get a generator, not an iterator." A generator is an instance of the generator class. As I understand the patch, chain would still produce chain objects, not generator objects, and hence would fail isinstance tests.

I suggested on the thread above that gentools should be a separate module, initially released on pypi. Unless it is easier than I think to produce generator instances in C, it should be coded in Python. That would be good anyway while developing the interface and behavior. Start with your generator chain function.

Perhaps it would help you to understand my viewpoint by considering the analogous statement "I think it would be useful to chain multiple files and still get a file, not an iterator.". Or substitute any other augmented iterator class for 'file'.
Date User Action Args
2013-03-28 01:02:27terry.reedylinkissue17433 superseder
2012-10-09 03:08:20terry.reedysetnosy: + terry.reedy
messages: + msg172438
2012-10-09 03:01:10ncoghlansetstatus: open -> closed
resolution: rejected
messages: + msg172437

stage: patch review -> resolved
2012-10-09 02:48:31gvanrossumsetmessages: + msg172436
2012-10-08 22:43:21gvanrossumsetmessages: + msg172423
2012-10-08 22:42:00gvanrossumsetnosy: + gvanrossum
messages: + msg172422
2012-10-07 07:49:01pyossetfiles: - itertools-chain-send-throw-and-close.diff
2012-10-07 07:48:53pyossetfiles: + itertools-chain-send-throw-and-close-2.diff

messages: + msg172280
2012-10-06 17:04:11serhiy.storchakasetnosy: + serhiy.storchaka
2012-10-06 15:00:58pyossetfiles: + itertools-chain-doc.diff
2012-10-06 14:56:08pyossetversions: - Python 3.3, Python 3.5
2012-10-06 14:55:48pyossetversions: + Python 3.3, Python 3.5
2012-10-06 14:53:28pitrousetnosy: + ncoghlan
stage: patch review
type: enhancement

versions: - Python 3.3, Python 3.5
2012-10-06 14:52:28pyoscreate