This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: sending non-None values makes generator raise StopIteration on next access
Type: behavior Stage: resolved
Components: Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, Zabolekar, brandtbucher, christian.heimes, pablogsal
Priority: release blocker Keywords: patch

Created on 2021-12-07 21:53 by Zabolekar, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29986 merged Mark.Shannon, 2021-12-08 11:01
PR 29988 merged Mark.Shannon, 2021-12-08 12:14
PR 30367 merged brandtbucher, 2022-01-03 19:41
Messages (7)
msg407975 - (view) Author: Stanislav Syekirin (Zabolekar) Date: 2021-12-07 21:53
Tested with Python 3.10.1 on Linux and Python 3.10.0 on Windows.

The following code prints None in 3.9 and raises StopIteration without any additional information in 3.10:

    def f():
        yield
    
    x = f()
    
    try:
        x.send(0)
    except TypeError as e:
        print(e) # can't send non-None value to a just-started generator
    
    print(next(x))
msg408013 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-12-08 12:09
New changeset 69806b9516dbe092381f3ef884c7c64bb9b8414a by Mark Shannon in branch 'main':
bpo-46009: Do not exhaust generator when send() method raises (GH-29986)
https://github.com/python/cpython/commit/69806b9516dbe092381f3ef884c7c64bb9b8414a
msg408021 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-12-08 14:46
New changeset 99c72326d245fb604609a87a51ef1ad0845467b7 by Mark Shannon in branch '3.10':
[3.10] bpo-46009: Do not exhaust generator when send() method raises (GH-29986). (GH-29988)
https://github.com/python/cpython/commit/99c72326d245fb604609a87a51ef1ad0845467b7
msg409700 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2022-01-04 19:38
New changeset 31e43cbe5f01cdd5b5ab330ec3040920e8b61a91 by Brandt Bucher in branch 'main':
bpo-46009: Remove GEN_START (GH-30367)
https://github.com/python/cpython/commit/31e43cbe5f01cdd5b5ab330ec3040920e8b61a91
msg409770 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-01-05 15:05
GH-30367 broke Emscripten WASM builds. I'm getting a "null function or function signature mismatch" error from the WASM engine:

RuntimeError: null function or function signature mismatch
    at _PyEval_EvalFrameDefault (http://localhost:8000/python.wasm:wasm-function[2383]:0x103a6f)
    at gen_send_ex2 (http://localhost:8000/python.wasm:wasm-function[886]:0x4f35d)
    at gen_iternext (http://localhost:8000/python.wasm:wasm-function[893]:0x4fb1a)
    at builtin_any (http://localhost:8000/python.wasm:wasm-function[3927]:0x198a87)
    at cfunction_vectorcall_O (http://localhost:8000/python.wasm:wasm-function[403]:0x201ae)
    at PyObject_Vectorcall (http://localhost:8000/python.wasm:wasm-function[2426]:0x109dbe)
    at _PyEval_EvalFrameDefault (http://localhost:8000/python.wasm:wasm-function[2383]:0x1062ba)
    at _PyEval_Vector (http://localhost:8000/python.wasm:wasm-function[2380]:0xfc366)
    at PyEval_EvalCode (http://localhost:8000/python.wasm:wasm-function[2379]:0xfc014)
    at _PyConfig_InitPathConfig (http://localhost:8000/python.wasm:wasm-function[6871]:0x2924b0)

I'm trying to get more debug information now.
msg409776 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-01-05 15:37
I got more debug symbols:

python.js:235 Uncaught RuntimeError: null function or function signature mismatch
    at _PyEval_EvalFrameDefault (ceval.c:4247)
    at _PyEval_EvalFrame (pycore_ceval.h:48)
    at gen_send_ex2 (genobject.c:219)
    at gen_iternext (genobject.c:583)
    at builtin_any (bltinmodule.c:384)
    at cfunction_vectorcall_O (methodobject.c:516)
    at _PyObject_VectorcallTstate.11 (pycore_call.h:89)
    at PyObject_Vectorcall (call.c:298)
    at _PyEval_EvalFrameDefault (ceval.c:4625)
    at _PyEval_EvalFrame.1 (pycore_ceval.h:48)

The culprit is 

   PyObject *next = (*Py_TYPE(iter)->tp_iternext)(iter);

in the TARGET(FOR_ITER) target. An additional NULL check resolves the problem.


index 953876f6226..390400b3246 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -4244,7 +4244,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr
             PREDICTED(FOR_ITER);
             /* before: [iter]; after: [iter, iter()] *or* [] */
             PyObject *iter = TOP();
-            PyObject *next = (*Py_TYPE(iter)->tp_iternext)(iter);
+            PyObject *(*iternext)(PyObject *) = *Py_TYPE(iter)->tp_iternext;
+            if (iternext == NULL) {
+                PyErr_BadInternalCall();
+                goto error;
+               }
+            PyObject *next = iternext(iter);
             if (next != NULL) {
                 PUSH(next);
                 PREDICT(STORE_FAST);
msg409781 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-01-05 16:02
False alarm ... the frozen files for getpath.py were out of date. I can no longer reproduce the issue after a hard git clean and rebuild.
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90167
2022-01-05 16:02:13christian.heimessetstatus: open -> closed
resolution: fixed
messages: + msg409781

stage: needs patch -> resolved
2022-01-05 15:37:16christian.heimessetversions: + Python 3.11
2022-01-05 15:37:00christian.heimessetmessages: + msg409776
2022-01-05 15:05:20christian.heimessetstatus: closed -> open

nosy: + christian.heimes
messages: + msg409770

resolution: fixed -> (no value)
stage: resolved -> needs patch
2022-01-04 19:38:36brandtbuchersetmessages: + msg409700
2022-01-03 19:41:44brandtbuchersetnosy: + brandtbucher

pull_requests: + pull_request28580
2021-12-08 14:53:31pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-12-08 14:46:40pablogsalsetmessages: + msg408021
2021-12-08 12:14:59Mark.Shannonsetpull_requests: + pull_request28211
2021-12-08 12:09:46Mark.Shannonsetmessages: + msg408013
2021-12-08 11:01:20Mark.Shannonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request28209
2021-12-07 23:45:13pablogsalsetpriority: normal -> release blocker
nosy: + pablogsal
2021-12-07 23:44:29pablogsalsetnosy: + Mark.Shannon
2021-12-07 21:53:29Zabolekarcreate