classification
Title: Backport next()
Type: behavior Stage:
Components: Interpreter Core Versions: Python 2.6
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: gvanrossum Nosy List: belopolsky, georg.brandl, gvanrossum, rhettinger
Priority: critical Keywords: 26backport, patch

Created on 2008-04-29 21:02 by georg.brandl, last changed 2008-04-30 19:47 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
nextbackport.diff georg.brandl, 2008-04-29 21:02 #1
Messages (13)
msg65980 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-04-29 21:01
Backporting 3.0's next() builtin.

There's no change w.r.t. __next__/next, that is tracked in #2336.
msg65992 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2008-04-30 06:04
ISTM, the only value added by next(g) is that it replaces g.next() with 
a more conventional spelling, g.__next__().  Since 2.6 still has g.next
(),I don't see how this backport adds value.  It does however create a 
second way to do it that will be confusing to some remaining in the 2.x 
world.  I think we should avoid double spellings in 2.6 except in cases 
where the 2-to-3 converter would need help.
msg65993 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-04-30 06:29
IMO having next() in 2.6 helps since if you use it consistently you
don't have to care about calling .next() or .__next__().

Also, I don't see how this is different from having e.g. reduce() and
functools.reduce() in 2.6.
msg65995 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2008-04-30 06:37
The problem is with the "if you use it consistently" premise.  That 
will not hold in an environment with legacy code, multiple programmers, 
lots of code in ASPN recipes and published materials, and third-party 
modules.  A patch like this dooms Py2.6 programmers to seeing both of 
these forms intermixed throughout the code base.  This is *not* a win.
msg66000 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-04-30 13:39
I think it's important to make this available in 2.6; it will let people
writing 3.0-oriented code in 2.6 (as several developers are planning to
do) do the right thing for this syntax.
msg66004 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-04-30 15:01
I thought new code is supposed to use Py_TYPE macro instead of ->ob_type:

+			"%.200s object is not an iterator", it->ob_type-
>tp_name);
..
+	res = (*it->ob_type->tp_iternext)(it);

Py3k branch has the same issue.
msg66005 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-04-30 15:20
One more question: What is the rationale for

+	res = (*it->ob_type->tp_iternext)(it);
+	if (res == NULL) {
..
+			PyErr_SetNone(PyExc_StopIteration);
+			return NULL;

?

I would think tp_iternext failing to set an exception should not be 
translated into stop iteration.  Instead, builtin_next() should return 
NULL without an exception set and thus trigger a SystemError.
msg66006 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-04-30 16:40
> I would think tp_iternext failing to set an exception should not be 
> translated into stop iteration.  Instead, builtin_next() should return 
> NULL without an exception set and thus trigger a SystemError.

Wrong; the iternext slot is designed to return NULL without setting an
exception.  See e.g. listiter_next().
msg66007 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-04-30 16:48
+1 on this. I have a few nits about the code:

Line 1083: "%.200s object is not an iterator", it->ob_type->tp_name);
Line is too long.

Line 1088: if (res == NULL) {
How about

  if (res != NULL)
    return res;

?

Line 1089: if (def) {
if (def != NULL) {

Line 1093: PyErr_Clear();
I would only call this if PyErr_Occurred() returns true.
msg66008 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-04-30 16:54
On Wed, Apr 30, 2008 at 12:41 PM, Guido van Rossum
<report@bugs.python.org> wrote:
>
>  Guido van Rossum <guido@python.org> added the comment:
>
>  > ..  builtin_next() should return
>  > NULL without an exception set and thus trigger a SystemError.
>
>  Wrong; the iternext slot is designed to return NULL without setting an
>  exception.  See e.g. listiter_next().

I did not know that.  Thanks for the explanation.  In this case,
wouldn't it be cleaner to call PyIter_Next which is documented to
return NULL with no exception?
msg66009 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-04-30 17:13
On Wed, Apr 30, 2008 at 12:41 PM, Guido van Rossum
<report@bugs.python.org> wrote:
> the iternext slot is designed to return NULL without setting an
>  exception.

This is not what the documentation says:

"""
iternextfunc PyTypeObject.tp_iternext
An optional pointer to a function that returns the next item in an
iterator, or raises StopIteration when the iterator is exhausted.
""" <http://docs.python.org/dev/c-api/typeobj.html#tp_iternext>

It looks like documentation needs to be updated, but wouldn't it be
odd to specify that setting StopIteration exception is optional?  It's
probably more logical to intercept StopIteration in slot_tp_iternext
rather than at every place where tp_iternext is called.
msg66010 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2008-04-30 17:22
Feel free to submit a patch to fix the docs.

Changing the API is not an option -- it's been like this since the
tp_iternext slot was added, and it's been designed like this for a
reason: so that in the common case of iterating over a built-in sequence
type no exception objects have to be created.  In particular the
for-loop code would just discard the StopIteration instance again.

The requirement that the exception is *optional* is so that if you're
calling a Python iterator that *does* create the exception, the
exception object (with whatever data the creator might have attached to
it) doesn't get lost (or worse, have to be recreated).

Calling PyIter_Next() here instead of inlining it would not be
advantageous; it would just slow things down since we'd have to make a
redundant call to PyErr_Occurred() to distinguish the StopIteration case
from other errors.
msg66017 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-04-30 19:47
Updated and committed as r62599. Also fixed your nits in the original 3k
version in r62598.
History
Date User Action Args
2008-04-30 19:47:35georg.brandlsetstatus: open -> closed
messages: + msg66017
2008-04-30 17:22:35gvanrossumsetmessages: + msg66010
2008-04-30 17:13:49belopolskysetmessages: + msg66009
2008-04-30 16:54:02belopolskysetmessages: + msg66008
2008-04-30 16:48:50gvanrossumsetresolution: accepted
messages: + msg66007
2008-04-30 16:41:00gvanrossumsetmessages: + msg66006
2008-04-30 15:20:57belopolskysetmessages: + msg66005
2008-04-30 15:01:26belopolskysetnosy: + belopolsky
messages: + msg66004
2008-04-30 13:39:37gvanrossumsetmessages: + msg66000
2008-04-30 06:37:23rhettingersetmessages: + msg65995
2008-04-30 06:29:57georg.brandlsetmessages: + msg65993
2008-04-30 06:04:31rhettingersetnosy: + rhettinger
messages: + msg65992
2008-04-29 21:02:26georg.brandlcreate