classification
Title: async-for loops are traced incorrectly in Python 3.10
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, jcea, lukasz.langa, nedbat, pablogsal
Priority: release blocker Keywords: 3.10regression, patch

Created on 2021-07-13 13:03 by nedbat, last changed 2021-07-15 14:56 by lukasz.langa. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27160 merged Mark.Shannon, 2021-07-15 10:12
PR 27163 merged Mark.Shannon, 2021-07-15 13:50
Messages (4)
msg397396 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-07-13 13:03
In Python 3.10, the traces at the end of an async-for loop are incorrect and different than at the end of a for-loop.

------------------------------
import linecache, sys

def trace(frame, event, arg):
    # The weird globals here is to avoid a NameError on shutdown...
    if frame.f_code.co_filename == globals().get("__file__"):
        lineno = frame.f_lineno
        print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, lineno).rstrip()))
    return trace

import asyncio

class AsyncIter:
    def __init__(self, items): self.items = items

    async def __aiter__(self):
        for i in self.items: yield i

async def test1():
    async for i in AsyncIter([1]):
        print(f"test1 {i}")

def test2():
    for i in [1]:
        print(f"test2 {i}")

print(sys.version)
sys.settrace(trace)

asyncio.run(test1())
test2()
------------------------------------

In 3.7, 3.8 and 3.9, the two for loops behave the same: the loop jumps back to the "for" statement, and then returns (the arrowed lines):

  3.9.5 (default, May  5 2021, 06:50:43)
  [Clang 12.0.0 (clang-1200.0.32.29)]
  call 18: async def test1():
  line 19:     async for i in AsyncIter([1]):
  call 13:     def __init__(self, items): self.items = items
  line 13:     def __init__(self, items): self.items = items
  retu 13:     def __init__(self, items): self.items = items
  call 15:     async def __aiter__(self):
  line 16:         for i in self.items: yield i
  retu 16:         for i in self.items: yield i
  exce 19:     async for i in AsyncIter([1]):
  line 20:         print(f"test1 {i}")
  test1 1
  line 19:     async for i in AsyncIter([1]):
  call 16:         for i in self.items: yield i
  line 16:         for i in self.items: yield i
  retu 16:         for i in self.items: yield i
  exce 19:     async for i in AsyncIter([1]):
> retu 19:     async for i in AsyncIter([1]):
  call 22: def test2():
  line 23:     for i in [1]:
  line 24:         print(f"test2 {i}")
  test2 1
  line 23:     for i in [1]:
> retu 23:     for i in [1]:


In 3.10, the for loop behaves the same, but now the async-for traces the body once more when it doesn't execute, and returns from the body of the loop (the starred line):

  3.10.0b4 (default, Jul 11 2021, 13:51:53) [Clang 12.0.0 (clang-1200.0.32.29)]
  call 18: async def test1():
  line 19:     async for i in AsyncIter([1]):
  call 13:     def __init__(self, items): self.items = items
  line 13:     def __init__(self, items): self.items = items
  retu 13:     def __init__(self, items): self.items = items
  call 15:     async def __aiter__(self):
  line 16:         for i in self.items: yield i
  retu 16:         for i in self.items: yield i
  exce 19:     async for i in AsyncIter([1]):
  line 20:         print(f"test1 {i}")
  test1 1
  line 19:     async for i in AsyncIter([1]):
  call 16:         for i in self.items: yield i
  line 16:         for i in self.items: yield i
  retu 16:         for i in self.items: yield i
  exce 19:     async for i in AsyncIter([1]):
* line 20:         print(f"test1 {i}")
  retu 20:         print(f"test1 {i}")
  call 22: def test2():
  line 23:     for i in [1]:
  line 24:         print(f"test2 {i}")
  test2 1
  line 23:     for i in [1]:
> retu 23:     for i in [1]:
msg397510 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-14 22:10
Marking as a release blocker. Mark, please take a look at your earliest convenience
msg397550 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-07-15 13:38
New changeset f333ab0f2edec26a769ed558263ac662e5475451 by Mark Shannon in branch 'main':
bpo-44622: Set line number of END_ASYNC_FOR to match that of iterator. (GH-27160)
https://github.com/python/cpython/commit/f333ab0f2edec26a769ed558263ac662e5475451
msg397556 - (view) Author: Ɓukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-15 14:54
New changeset 47695e3c88343e794d37333853b2ba3f16505c5d by Mark Shannon in branch '3.10':
bpo-44622: Set line number of END_ASYNC_FOR to match that of iterator. (GH-27160) (GH-27163)
https://github.com/python/cpython/commit/47695e3c88343e794d37333853b2ba3f16505c5d
History
Date User Action Args
2021-07-15 14:56:21lukasz.langasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-07-15 14:54:46lukasz.langasetnosy: + lukasz.langa
messages: + msg397556
2021-07-15 13:50:12Mark.Shannonsetpull_requests: + pull_request25700
2021-07-15 13:38:04Mark.Shannonsetmessages: + msg397550
2021-07-15 13:31:22jceasetnosy: + jcea
2021-07-15 10:12:43Mark.Shannonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25697
2021-07-14 22:10:13pablogsalsetmessages: + msg397510
2021-07-14 22:09:53pablogsalsetpriority: normal -> release blocker
nosy: + pablogsal
2021-07-13 13:03:51nedbatcreate