classification
Title: Deprecate creation of asyncio object when the loop is not running
Type: Stage:
Components: asyncio Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, callumquick, eamanu, lschoe, yselivanov
Priority: normal Keywords: patch

Created on 2019-10-27 07:08 by asvetlov, last changed 2019-11-06 22:31 by eamanu.

Files
File name Uploaded Description Edit
0001-Add-a-deprectation-warning-for-queue-w-o-event-runni.patch eamanu, 2019-11-06 22:31
Messages (11)
msg355449 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 07:08
Consider the following code:

import asyncio

q = asyncio.Queue()

async def main():
    await asyncio.gather(q.put(1), q.get(1))

asyncio.run(main())


This code just hangs since run() creates a loop but queue is bound with another (default) event loop.

The error is confusing and hard-to-debug.

We should raise a warning at least for the case; start from DeprecationWarning and make the system stricter later.

asyncio/locks.py is also affected since it has first-class classes (classes that instantiated by a user directly without factories).
msg355486 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-10-27 17:29
Yes.

As a remedy for this particular problem we can add checks here and there comparing `asyncio.get_running_loop()` and `self._loop`.  Performance will suffer a bit but the usability will greatly improve.
msg355507 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 23:55
I have a different feeling: we should start raising deprecation for asyncio.Queue() if it is created without running event loop.

It required `get_event_loop()` and `loop.is_running()` checks.

Later the pair can be replaced with `get_running_loop()`.

I think no check at awaiting time is needed; we never do it for other asyncio parts.  Just control of the loop at creation time is enough; if the object is created inside a running loop the sane code will use it with this loop context always.
msg355508 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 23:55
At least it is my learned lesson from aiohttp development.
msg355588 - (view) Author: Berry Schoenmakers (lschoe) Date: 2019-10-28 18:54
The current implementation of asyncio.run() is focused quite a bit on one-shot use. After the call returns, the default event loop is even gone: calling asyncio.get_event_loop() gives "RuntimeError: There is no current event loop in thread 'MainThread'."

It would be nice if asyncio.run() uses the default loop if it's available (and not running), and that the default loop remains intact after the call returns. 

This way multiple calls to asyncio.run() that all use the default loop are supported, maybe using an asyncio.Queue (or whatever) across these calls---attaching everything to the same (default) loop.

I find this very useful, for instance when writing unit tests.
msg355589 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-10-28 19:07
> It would be nice if asyncio.run() uses the default loop if it's available (and not running), and that the default loop remains intact after the call returns. 

Unfortunately it's not possible to implement that reliably and without a bunch of surprising behaviors.

Also, if you want the loop to be intact after asyncio.run, it means that you would not want to close it;that would defeat the purpose of asyncio.run.
msg355597 - (view) Author: Berry Schoenmakers (lschoe) Date: 2019-10-28 22:19
Well, I'm basically using a run method defined as a shorthand for self.loop.run_until_complete (without closing loop, reusing it throughout). It would be nice if asyncio.run could simply be used instead, but I understand you're saying this is easier said than done, which is fine with me. Thanks.
msg355598 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-10-28 22:23
> Well, I'm basically using a run method defined as a shorthand for self.loop.run_until_complete (without closing loop, reusing it throughout). It would be nice if asyncio.run could simply be used instead, but I understand you're saying this is easier said than done, which is fine with me. Thanks.

Yes. `asyncio.run()` is similar to `loop.run_until_complete()` but adds some guarantees + sane setup/cleanup. If we strip proper cleanup then there's no point in `asyncio.run()`.  So using `loop.run_until_complete()` is totally fine when you want to preserve the event loop between runs.
msg355601 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-28 22:59
I'm pretty happy with asyncio.run() functionalit.

I used run() for the bug demonstration, but the example can be rewritten easily without this function.

The problem is not in run() but in an object lifecycle. Implicit loop creation plus module-level initialization provides a pretty big set of ways to shoot in the foot :)
msg355944 - (view) Author: Callum Ward (callumquick) * Date: 2019-11-04 14:33
Hi, I'm a new contributor and this issue looks like an interesting enhancement: is there any consensus forming on what we want to limit to e.g. raising depreciation warnings when these first-class classes are created without running event loop?
msg356157 - (view) Author: Emmanuel Arias (eamanu) * Date: 2019-11-06 22:31
Hi, 

@asvetlov are you thinking something like the patch attached?
History
Date User Action Args
2019-11-06 22:31:38eamanusetfiles: + 0001-Add-a-deprectation-warning-for-queue-w-o-event-runni.patch
keywords: + patch
messages: + msg356157
2019-11-06 21:48:10eamanusetnosy: + eamanu
2019-11-04 14:33:21callumquicksetnosy: + callumquick
messages: + msg355944
2019-10-28 22:59:07asvetlovsetmessages: + msg355601
2019-10-28 22:23:41yselivanovsetmessages: + msg355598
2019-10-28 22:19:49lschoesetmessages: + msg355597
2019-10-28 19:07:49yselivanovsetmessages: + msg355589
2019-10-28 18:54:43lschoesetnosy: + lschoe
messages: + msg355588
2019-10-27 23:55:44asvetlovsetmessages: + msg355508
2019-10-27 23:55:17asvetlovsetmessages: + msg355507
2019-10-27 17:29:48yselivanovsetmessages: + msg355486
2019-10-27 07:08:05asvetlovcreate