This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Deprecate creation of asyncio object when the loop is not running
Type: Stage: resolved
Components: asyncio Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder: [document removal of] the deprecated 'loop' parameter asyncio API in 3.10
View: 42392
Assigned To: Nosy List: achimnol, asvetlov, callumquick, eamanu, lschoe, yselivanov
Priority: normal Keywords: patch

Created on 2019-10-27 07:08 by asvetlov, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
0001-Add-a-deprectation-warning-for-queue-w-o-event-runni.patch eamanu, 2019-11-06 22:31
Pull Requests
URL Status Linked Edit
PR 18195 closed eamanu, 2020-01-26 23:49
Messages (13)
msg355449 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 07:08
Consider the following code:

import asyncio

q = asyncio.Queue()

async def main():
    await asyncio.gather(q.put(1), q.get(1))

asyncio.run(main())


This code just hangs since run() creates a loop but queue is bound with another (default) event loop.

The error is confusing and hard-to-debug.

We should raise a warning at least for the case; start from DeprecationWarning and make the system stricter later.

asyncio/locks.py is also affected since it has first-class classes (classes that instantiated by a user directly without factories).
msg355486 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-10-27 17:29
Yes.

As a remedy for this particular problem we can add checks here and there comparing `asyncio.get_running_loop()` and `self._loop`.  Performance will suffer a bit but the usability will greatly improve.
msg355507 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 23:55
I have a different feeling: we should start raising deprecation for asyncio.Queue() if it is created without running event loop.

It required `get_event_loop()` and `loop.is_running()` checks.

Later the pair can be replaced with `get_running_loop()`.

I think no check at awaiting time is needed; we never do it for other asyncio parts.  Just control of the loop at creation time is enough; if the object is created inside a running loop the sane code will use it with this loop context always.
msg355508 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 23:55
At least it is my learned lesson from aiohttp development.
msg355588 - (view) Author: Berry Schoenmakers (lschoe) * Date: 2019-10-28 18:54
The current implementation of asyncio.run() is focused quite a bit on one-shot use. After the call returns, the default event loop is even gone: calling asyncio.get_event_loop() gives "RuntimeError: There is no current event loop in thread 'MainThread'."

It would be nice if asyncio.run() uses the default loop if it's available (and not running), and that the default loop remains intact after the call returns. 

This way multiple calls to asyncio.run() that all use the default loop are supported, maybe using an asyncio.Queue (or whatever) across these calls---attaching everything to the same (default) loop.

I find this very useful, for instance when writing unit tests.
msg355589 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-10-28 19:07
> It would be nice if asyncio.run() uses the default loop if it's available (and not running), and that the default loop remains intact after the call returns. 

Unfortunately it's not possible to implement that reliably and without a bunch of surprising behaviors.

Also, if you want the loop to be intact after asyncio.run, it means that you would not want to close it;that would defeat the purpose of asyncio.run.
msg355597 - (view) Author: Berry Schoenmakers (lschoe) * Date: 2019-10-28 22:19
Well, I'm basically using a run method defined as a shorthand for self.loop.run_until_complete (without closing loop, reusing it throughout). It would be nice if asyncio.run could simply be used instead, but I understand you're saying this is easier said than done, which is fine with me. Thanks.
msg355598 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-10-28 22:23
> Well, I'm basically using a run method defined as a shorthand for self.loop.run_until_complete (without closing loop, reusing it throughout). It would be nice if asyncio.run could simply be used instead, but I understand you're saying this is easier said than done, which is fine with me. Thanks.

Yes. `asyncio.run()` is similar to `loop.run_until_complete()` but adds some guarantees + sane setup/cleanup. If we strip proper cleanup then there's no point in `asyncio.run()`.  So using `loop.run_until_complete()` is totally fine when you want to preserve the event loop between runs.
msg355601 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-28 22:59
I'm pretty happy with asyncio.run() functionalit.

I used run() for the bug demonstration, but the example can be rewritten easily without this function.

The problem is not in run() but in an object lifecycle. Implicit loop creation plus module-level initialization provides a pretty big set of ways to shoot in the foot :)
msg355944 - (view) Author: Callum Ward (callumquick) * Date: 2019-11-04 14:33
Hi, I'm a new contributor and this issue looks like an interesting enhancement: is there any consensus forming on what we want to limit to e.g. raising depreciation warnings when these first-class classes are created without running event loop?
msg356157 - (view) Author: Emmanuel Arias (eamanu) * Date: 2019-11-06 22:31
Hi, 

@asvetlov are you thinking something like the patch attached?
msg359410 - (view) Author: Joongi Kim (achimnol) * Date: 2020-01-06 09:30
It is also generating deprecation warning:

> /opt/python/3.8.0/lib/python3.8/asyncio/queues.py:48: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.
>   self._finished = locks.Event(loop=loop)
msg381992 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2020-11-28 11:09
Fixed by #42392
History
Date User Action Args
2022-04-11 14:59:22adminsetgithub: 82780
2020-11-28 11:09:52asvetlovsetstatus: open -> closed
versions: + Python 3.10, - Python 3.9
superseder: [document removal of] the deprecated 'loop' parameter asyncio API in 3.10
messages: + msg381992

resolution: fixed
stage: patch review -> resolved
2020-01-26 23:49:35eamanusetstage: patch review
pull_requests: + pull_request17576
2020-01-06 09:30:34achimnolsetnosy: + achimnol
messages: + msg359410
2019-11-06 22:31:38eamanusetfiles: + 0001-Add-a-deprectation-warning-for-queue-w-o-event-runni.patch
keywords: + patch
messages: + msg356157
2019-11-06 21:48:10eamanusetnosy: + eamanu
2019-11-04 14:33:21callumquicksetnosy: + callumquick
messages: + msg355944
2019-10-28 22:59:07asvetlovsetmessages: + msg355601
2019-10-28 22:23:41yselivanovsetmessages: + msg355598
2019-10-28 22:19:49lschoesetmessages: + msg355597
2019-10-28 19:07:49yselivanovsetmessages: + msg355589
2019-10-28 18:54:43lschoesetnosy: + lschoe
messages: + msg355588
2019-10-27 23:55:44asvetlovsetmessages: + msg355508
2019-10-27 23:55:17asvetlovsetmessages: + msg355507
2019-10-27 17:29:48yselivanovsetmessages: + msg355486
2019-10-27 07:08:05asvetlovcreate