This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: AsyncIterator on 3.7: __aiter__ no longer honors finally blocks
Type: behavior Stage:
Components: asyncio Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asksol, asvetlov, ned.deily, twisteroid ambassador, xtreak, yselivanov
Priority: normal Keywords: 3.7regression

Created on 2019-03-23 00:15 by asksol, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
test_iterators_and_finally_blocks.py asksol, 2019-03-23 00:15
Messages (7)
msg338630 - (view) Author: Ask Solem (asksol) (Python committer) Date: 2019-03-23 00:15
We use finally blocks in ``__aiter__`` like this:

```
class AsyncFinallyIterator:

   def __aiter__(self):
       for i in range(10):
           try:
               yield i
           finally:
               print('FINALLY')
```

Attached is a test for both iterators and async iterators.
The tests pass on Python 3.6, but only the non-async iterator test pass under Python 3.7.

Thanks for your attention!


This worked perfectly well in Python 3.6, but stopped working in Python 3.7.

I also verified that Iterator supports the same construct (and this works in both Python 3.6 and 3.7):

```

class FinallyIterator:

    def __iter__(self):
        for i in range(10):
            try:
                yield i
            finally:
                print('FINALLY')
```
msg338638 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-03-23 01:02
The attached script fails on master too. On bisecting could this be possibly caused due to 41e5ec377b (bpo-34769) ? Tagging Ned since it was introduced from 3.7.1rc2 and also backported to 3.6 .

➜  cpython git:(41e5ec377b) git checkout 41e5ec377b && make -s -j4 > /dev/null
HEAD is now at 41e5ec377b bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716)
➜  cpython git:(41e5ec377b) ./python.exe ../backups/bpo36403.py
F.
======================================================================
FAIL: test_main (__main__.TestAsyncIteratorFinally)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../backups/bpo36403.py", line 30, in test_main
    self.assertTrue(it.finally_executed)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 2 tests in 0.002s

FAILED (failures=1)
➜  cpython git:(41e5ec377b) git checkout 41e5ec377b~1 && make -s -j4 > /dev/null
Previous HEAD position was 41e5ec377b bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716)
HEAD is now at 0ce31d340b bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (GH-9656)
➜  cpython git:(0ce31d340b) ./python.exe ../backups/bpo36403.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.003s

OK
msg338642 - (view) Author: Ask Solem (asksol) (Python committer) Date: 2019-03-23 05:35
Ah, so the extra call_soon means it needs a:

[code]
loop.run_until_complete(asyncio.sleep(0))```
[/code]

before the self.assertTrue(it.finally_executed)

to finish executing agen.close().

Why is create_task different? Does it execute an iteration of the generator immediately?

Seems good for this behavior to be consistent, but not sure how difficult that would be.
msg338647 - (view) Author: Ask Solem (asksol) (Python committer) Date: 2019-03-23 06:29
Perhaps we could add a self._finally to the event loop itself?
Like loop._ready, but a list of callbacks run_until_complete will call before returning?
msg345805 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-17 07:34
Ping.  This was marked as a 3.7regression.  Is a change needed?
msg345808 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-06-17 07:47
From my understanding yield inside __aiter__ is a forbidden construction. Even if parser allows it the statement doesn't make any sense.

I'm very interested to hear Yuri opinion.
msg371341 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-06-12 08:32
I note this is marked as a 3.7regression and still open. Since the cutoff for the final 3.7 bugfix mode release is in a few days, I'm assuming this means that 3.7 users will have to live with this regression.  If you feel that is a problem, speak up now.
History
Date User Action Args
2022-04-11 14:59:12adminsetgithub: 80584
2020-06-12 08:32:55ned.deilysetmessages: + msg371341
2019-06-17 07:47:23asvetlovsetmessages: + msg345808
2019-06-17 07:34:15ned.deilysetmessages: + msg345805
2019-03-23 06:29:03asksolsetmessages: + msg338647
2019-03-23 05:35:34asksolsetmessages: + msg338642
2019-03-23 01:02:15xtreaksetversions: + Python 3.8
nosy: + ned.deily, twisteroid ambassador, asvetlov, xtreak

messages: + msg338638

components: + asyncio
type: behavior
2019-03-23 00:24:52xtreaksetnosy: + yselivanov
2019-03-23 00:15:07asksolcreate