classification
Title: "Equivalent to" code for zip is wrong in Python 3
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.2, Python 3.1
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Douglas.Leeder, belopolsky, docs@python, max, orsenthil, rhettinger, stutzbach
Priority: normal Keywords:

Created on 2010-10-05 18:26 by max, last changed 2010-10-10 07:10 by rhettinger. This issue is now closed.

Messages (14)
msg118025 - (view) Author: Max (max) Date: 2010-10-05 18:26
The sample code explaining zip function is incorrect at http://docs.python.org/py3k/library/functions.html?highlight=zip#zip:

def zip(*iterables):
    # zip('ABCD', 'xy') --> Ax By
    iterables = map(iter, iterables)
    while iterables:
        yield tuple(map(next, iterables))

See http://stackoverflow.com/questions/3865640/understanding-zip-function for discussion.
msg118026 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-05 18:37
The relevant comment at Stack Overflow is:

"""
It looks like it's a bug in the documentation. The 'equivalent' code working in python2, but not in python3, where it has an infinite loop.

And the latest version of the documentation has the same problem: http://docs.python.org/release/3.1.2/library/functions.html

Look like change r61361 was the problem, as it merged changes from python 2.6 without verifying that they were correct for python 3.
"""
msg118027 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-10-05 18:51
The code was taken from the itertools.izip documentation for Python 2, where it did work.

In Python 2, next() raises StopIteration which is propagated up and causes the izip() to stop.  In Python 3, map is itself a generator and the StopIteration terminates the map operation instead of terminating the zip operation.
msg118028 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-05 18:57
Note that the following variant where maps are replaced with list comprehensions seems to work:

def zip(*iterables):
    # zip('ABCD', 'xy') --> Ax By                                                                                                                                                      
    iterables = [iter(i) for i in iterables]
    while iterables:
	yield tuple([next(j) for j in iterables])
msg118115 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-07 15:53
As Daniel pointed out, the "equivalent to" code in builtins section comes from 2.x itertools documentation where and equivalent generator definition is presented for each function.  While these definitions are helpful when used for documenting a module oriented towards more advanced users, I doubt that exposing novices who are looking up builtins to the yield keyword and generators is a good idea.  The zip() example is particularly problematic.  Conceptually, zip is a very simple function, but the "equivalent to" code is not easy to decipher.   The reliance on StopIteration exception escaping from map to break out of the infinite loop is clever, but not obvious.  Moreover, as this bug demonstrates, this trick relies on subtle details that changed in 3.x.

I suggest removing the "equivalent to" code from the zip section and replacing it with an example showing how to use zip with a for loop similar to the example illustrating enumerate.
msg118116 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-10-07 16:00
> I suggest removing the "equivalent to" code from the zip section and
> replacing it with an example showing how to use zip with a for loop
> similar to the example illustrating enumerate.

+1
msg118122 - (view) Author: Max (max) Date: 2010-10-07 17:10
Personally, I find it impossible in some cases to understand exactly what a function does just from reading a textual description. In those cases, I always refer to the equivalent code if it's given. In fact that's the reason I was looking going the zip equivalent function!

I would feel it's a loss if equivalent code disappear from the docs.

I understand sometimes the code requires maintenance, but I'd rather live with some temporary bugs than lose the equivalent code.

As to subtleties of how it works, that's not really a concern, if that's the only way to understand the precise meaning of whatever it explains.
msg118129 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-10-07 19:37
I'll update the docs with an equivalent that works and that has a comment showing when the StopIteration is raised and caught.
msg118211 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-08 18:25
On Thu, Oct 7, 2010 at 3:37 PM, Raymond Hettinger
<report@bugs.python.org> wrote:
..
> I'll update the docs with an equivalent that works and that has a comment showing when the
> StopIteration is raised and caught.
>

In this case, I wonder if "equivalent to" code should be added to the
section for enumerate() and map().  Also since any() and all() have
"equivalent to" code, I think min(), max() and sum() deserve it as
well.
msg118219 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-10-08 19:02
Refuse the temptation to hypergeneralize ;-)

Also refuse the temptation to double the size of the docs (more != better).

In the case of min/max, the pure python versions may add some value in showing that the first match is what is returned.  But the code will also be a bit convoluted because it needs paths for key-argument case and for the screwy interpretation of 1 arg vs multiple args.

The pure python code is there for any() and all() to show the early out feature and to suggest how you could roll-your-own if you want different behavior (such as returning the value of the first exception).
msg118300 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-10-10 03:49
On Fri, Oct 08, 2010 at 06:25:26PM +0000, Alexander Belopolsky wrote:
> In this case, I wonder if "equivalent to" code should be added to the
> section for enumerate() and map().  Also since any() and all() have
> "equivalent to" code, I think min(), max() and sum() deserve it as
> well.

I think, you are asking for consistency in docs, right?

As a sidenote those explanations should be okay, but merging it with
the explanation of the functions may add to confusion (like the zip
example did in this case). 

Even I am +1 on removing that complex equivalent code in the zip
documentation, as it can confusing to the newcomer. But I am also okay
with moving that 'equivalent to code' further down in the
explanation.
msg118306 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-10-10 05:57
Max, thanks for reporting this.  I've replaced the sample code, making it work correctly and more clearly showing the logic.

See r85345 and r85346.

Daniel, we need to sync-up on the meaning of marking a report as "accepted".  Traditionally it denotes an approved patch, not a agreement that the bug is valid.
msg118309 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-10-10 06:28
> Daniel, we need to sync-up on the meaning of marking a report as 
> "accepted".  Traditionally it denotes an approved patch, not a agreement > that the bug is valid.

Woops!  Thanks for the correction.  For what it's worth, a quick search of issues with Resolution: accepted and Stage: needs patch suggests that √Čric Araujo has perhaps been using the same misinterpretation as I have.  I'm not sure if I got the behavior from him, if he got it from me, or if we both arrived there independently. ;-)
msg118314 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-10-10 07:10
Please pass the word along if you get a chance :-)
History
Date User Action Args
2010-10-10 07:10:23rhettingersetmessages: + msg118314
2010-10-10 06:28:41stutzbachsetmessages: + msg118309
2010-10-10 05:58:20rhettingersetstatus: open -> closed
2010-10-10 05:57:12rhettingersetresolution: accepted -> fixed
messages: + msg118306
2010-10-10 03:49:45orsenthilsetnosy: + orsenthil
messages: + msg118300
2010-10-08 19:02:32rhettingersetmessages: + msg118219
2010-10-08 18:25:25belopolskysetmessages: + msg118211
2010-10-07 19:37:14rhettingersetmessages: + msg118129
2010-10-07 17:10:50maxsetmessages: + msg118122
2010-10-07 16:00:13stutzbachsetmessages: + msg118116
2010-10-07 15:53:53belopolskysetmessages: + msg118115
2010-10-06 09:01:20Douglas.Leedersetnosy: + Douglas.Leeder
2010-10-05 23:44:23belopolskysetnosy: - loewis
2010-10-05 18:57:37belopolskysetmessages: + msg118028
2010-10-05 18:51:56stutzbachsetversions: + Python 3.2

nosy: + stutzbach
title: bug in sample code in documentation -> "Equivalent to" code for zip is wrong in Python 3
messages: + msg118027
resolution: accepted
stage: needs patch
2010-10-05 18:43:10rhettingersetassignee: docs@python -> rhettinger
2010-10-05 18:38:02belopolskysetnosy: + rhettinger
2010-10-05 18:37:10belopolskysetnosy: + loewis, belopolsky
messages: + msg118026
2010-10-05 18:26:45maxcreate