Title: asyncio calls from sync/async, better docs or api support
Type: enhancement Stage: resolved
Components: asyncio Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7, Python 3.6
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Dima.Tisnek, asvetlov, blazespinnaker, yselivanov
Priority: normal Keywords:

Created on 2020-10-22 13:47 by blazespinnaker, last changed 2020-10-23 13:21 by blazespinnaker. This issue is now closed.

Messages (8)
msg379292 - (view) Author: Blaze Spinnaker (blazespinnaker) Date: 2020-10-22 13:47
In jupyter / ipython, other repls, as well as from libraries,  asyncio code can be called.  To simplify integration, there should be a way for libraries to transparently do the right thing, call await or start a new global running event loop.  

This can be done without breaking the colored functions design constraint (only async can call async) and would not lead to dynamic coroutine architectures like gevent / lua.

The change would significantly reduce confusion that is pervasive in the python ecosystem and has lead to many people using a defacto approach of monkey patching (a monkey patch which only works on the surface).

Alternative, if this approach simply can not be accepted, better and more emphasized headline asyncio documentation would be appropriate to explain why the constraint is in place and the best patterns to use to work around the problem.   

It took me a lot of googling before I understood the reasoning of what is a very significant design choice.

There are many instances of this issue causing problems, but let me highlight this link:
msg379295 - (view) Author: Blaze Spinnaker (blazespinnaker) Date: 2020-10-22 14:19
Note that the biggest problem are obviously repls, and there are quite a few (iPython has issues due to it's high coupling with jupyter).  Providing a contract for them will ensure consistent support across the ecosystem rather than a hodge podge of barely working hacks.

One solution, not meant to be prescriptive, is to provide a repl only API meant to execute at the top of the stack (from perspective of user) only.
msg379339 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2020-10-22 20:26
I'm not sure what you're actually proposing, and only have a vague understanding of what you're complaining about. This is perhaps related to If so then maybe leave a comment there?
msg379362 - (view) Author: Blaze Spinnaker (blazespinnaker) Date: 2020-10-22 21:44
asyncio code placed in a repl won't execute properly as it may (or may not) be running an event loop already.

This happens not just in traditional repls, but any platform that provides execution shells around code.  Other repls, Spyder, pyppeter, and a lot more.  Some of the maintainers reference running a monkey patch to fix the problem.

The link I provided referenced 40 complaints about this issue.

I am not really proposing anything, just that it be addressed by either new API or better documentation explaining how best to solve the problem.  I do provide an example of an idea.

But addressing is important.  Just leaving it up to the community is causing a lot of chaos, incompatibilities and bugs.   

Some are suggesting using get_event_loop() but you discourage using this api in the documentation.  Not sure what that guidance means exactly as it's not very clear under what constraints the API can be used and when it can't.
msg379364 - (view) Author: Blaze Spinnaker (blazespinnaker) Date: 2020-10-22 21:47
I guess what I'm saying, is you created a design problem with the sync can't call async constraint, but didn't suggest any solutions to solving it, and the solutions people are arriving at are pretty random.
msg379370 - (view) Author: Blaze Spinnaker (blazespinnaker) Date: 2020-10-22 22:03
btw, this is different than the other issue as I understand you don't want re-entrant calls to async.  Library integration can be done (painfully) by cherry picking the right API to call, but it's more painful with repls as that's not how they're used.
msg379393 - (view) Author: Dima Tisnek (Dima.Tisnek) * Date: 2020-10-23 01:52
Blaze, I think this needs a concrete proposal: either a patch, or at least a formal example how the new API would be used.

Note that there's also async-sig mailing list that's less formal, as well as to bounce ideas off others.
msg379433 - (view) Author: Blaze Spinnaker (blazespinnaker) Date: 2020-10-23 13:21
Ah, thanks.  Looked around for something like that but couldn't find it.  Will close this.
Date User Action Args
2020-10-23 13:21:20blazespinnakersetstatus: open -> closed
stage: resolved
2020-10-23 13:21:12blazespinnakersetmessages: + msg379433
2020-10-23 01:52:43Dima.Tisneksetnosy: + Dima.Tisnek
messages: + msg379393
2020-10-22 22:03:23blazespinnakersetmessages: + msg379370
2020-10-22 21:47:25blazespinnakersetmessages: + msg379364
2020-10-22 21:44:45blazespinnakersetmessages: + msg379362
2020-10-22 20:26:22yselivanovsetmessages: + msg379339
2020-10-22 14:19:41blazespinnakersetmessages: + msg379295
2020-10-22 14:08:31blazespinnakersetversions: + Python 3.6, Python 3.7, Python 3.8, Python 3.9, Python 3.10
2020-10-22 13:47:16blazespinnakercreate