classification
Title: Deprecate passing coroutine objects to asyncio.wait()
Type: behavior Stage:
Components: asyncio Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: aeros, asvetlov, miss-islington, rhettinger, serhiy.storchaka, xtreak, yselivanov
Priority: normal Keywords: patch

Created on 2018-09-24 16:42 by yselivanov, last changed 2019-10-29 04:32 by aeros.

Pull Requests
URL Status Linked Edit
PR 9543 merged yselivanov, 2018-09-24 16:48
PR 9577 merged miss-islington, 2018-09-25 18:51
PR 16975 merged aeros, 2019-10-29 00:07
PR 16977 open aeros, 2019-10-29 01:56
Messages (21)
msg326265 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-09-24 16:42
asyncio.wait() accepts coroutines, wraps them into Tasks, and later returns those implicitly created Tasks in (done, pending) sets.  This is very confusing to new asyncio users and it's almost impossible to figure out what is going on.  See the first PR to the docs for more info.

Andrew, I think we should deprecate passing coroutines to asyncio.wait() in 3.8, and disallow that in 4.0.
msg326271 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-24 17:06
4.0 is too far. Why not disallow them in 3.10?
msg326272 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-09-24 17:15
> 4.0 is too far. Why not disallow them in 3.10?

What's the current plan?  I thought it's going to be 3.8, 3.9, 4.0.  Is there a PEP detailing this?
msg326390 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-09-25 18:51
New changeset 996859a90df51f84eab47351702cb59c6db4428a by Yury Selivanov in branch 'master':
bpo-34790: [docs] Passing coroutines to asyncio.wait() can be confusing. (GH-9543)
https://github.com/python/cpython/commit/996859a90df51f84eab47351702cb59c6db4428a
msg326392 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-09-25 18:54
Actually, since Andrew also agrees that we need to deprecate passing coroutines to wait(), I'll keep this issue open until we add an actual DeprecationWarning in 3.8.
msg326393 - (view) Author: miss-islington (miss-islington) Date: 2018-09-25 18:57
New changeset 3cc9557d9f03086fa3b0ea166f8506087e6950e3 by Miss Islington (bot) in branch '3.7':
bpo-34790: [docs] Passing coroutines to asyncio.wait() can be confusing. (GH-9543)
https://github.com/python/cpython/commit/3cc9557d9f03086fa3b0ea166f8506087e6950e3
msg326395 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-09-25 19:05
PendingDeprecationWarning
msg326397 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-25 19:15
AFAIK there is no a plan for 4.0. But seems many core developers (including me) think that changing the major number should reflect some major changes like removing GIL. And Guido preferred 3.10 after 3.9.
msg355443 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-10-27 03:00
> Actually, since Andrew also agrees that we need to deprecate passing coroutines to wait(), I'll keep this issue open until we add an actual DeprecationWarning in 3.8.

Since 3.8 has been released and the deprecation notice is in the 3.8 whatsnew document, should we implement the warning in Lib/asyncio/tasks.py? If so, I can open a PR. 

> PendingDeprecationWarning

Also, it's not clear to me if this should be a DeprecationWarning or PendingDeprecationWarning. The most recent message from Yury in the issue suggests a PendingDeprecationWarning, but the actual entry in the documentation (https://docs.python.org/3/library/asyncio-task.html?highlight=asyncio%20wait#asyncio.wait) seems like it might imply that it would be a DeprecationWarning:

> Deprecated since version 3.8: If any awaitable in aws is a coroutine, it is automatically scheduled as a Task. Passing coroutines objects to wait() directly is deprecated as it leads to confusing behavior.
msg355448 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 07:01
I think we can add `DeprecationWarning` for 3.9.

Since it is a) just a warning b) already marked as deprecated in docs -- the harm is minimal.

Honestly, we just missed the issue when were prepared for 3.8
msg355450 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-10-27 07:25
> I think we can add `DeprecationWarning` for 3.9.

If we add the deprecation warning just for 3.9, would the removal release also be pushed forward? 

> Honestly, we just missed the issue when were prepared for 3.8

Yeah that's definitely understandable, there were quite a number of major changes made in 3.8. It was the first time I saw a deprecation clearly documented in a final release that didn't have an associated deprecation warning. 

As for 3.8, I would like to update the whatsnew entry to add a link to this bpo issue if that's okay, as it does not at the moment. I recently included the entry in a PR that added entries for multiple significant changes for asyncio in the 3.8, but I forgot to include the bpo link (mostly because I found it in the documentation, rather than from the bpo issue).
msg355451 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-10-27 07:32
> It was the first time I saw a deprecation clearly documented in a final release that didn't have an associated deprecation warning. 

I want to clarify that this may be a more common occurrence than I'm realizing, I'm not entirely certain. Also it's not intended at all as a criticism, it just surprised me since it was the first time I saw it happen.
msg355457 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 08:47
> If we add the deprecation warning just for 3.9, would the removal release also be pushed forward? 

Yes, deprecating in 3.9 with removal in 3.11 is fine.

Regarding 3.8 release notes update -- not sure if it is needed flr docs-only change.

In the current situation, we have so-called *soft deprecation*: bare coroutines are deprecated in docs without any code change. This is perfectly fine, we give our users extra time to prepare for changes.
msg355483 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-10-27 17:15
> Regarding 3.8 release notes update -- not sure if it is needed flr docs-only change.

I'm not certain if the entry is necessary; my main concern is just that it's already present in the 3.8 release notes/whatsnew without anywhere to look for further information. 

> In the current situation, we have so-called *soft deprecation*: bare coroutines are deprecated in docs without any code change. This is perfectly fine, we give our users extra time to prepare for changes.

So is something considered to be a "soft deprecation" if there is no code change? If so, I may have made a mistake by specifying a release in the entry I recently added for it (https://docs.python.org/3/whatsnew/3.8.html#deprecated):

"The explicit passing of coroutine objects to asyncio.wait() has been deprecated and will be removed in version 3.10."

I would propose to adjust this to:

"The explicit passing of coroutine objects to asyncio.wait() has been deprecated. (Contributed by Yury Selivanov in :issue:`34790`.)"

Then for 3.9 whatsnew, we could specify the release:

"The explicit passing of coroutine objects to asyncio.wait() has been deprecated and will be removed in 3.11."

Thoughts?
msg355487 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-10-27 17:32
I'd add `and will be removed in 3.11.` now.
msg355488 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-10-27 17:34
lgtm
msg355491 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-10-27 18:18
Sounds good, I'll work on opening a PR.
msg355603 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-10-29 00:10
GH-16975 Is a simple fix for the asyncio.wait() whatsnew entry for 3.8. I'll implement the deprecation warning and add a 3.9 whatsnew entry in a separate PR, since those changes won't be backported.
msg355605 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-10-29 01:59
GH-16977 Implements the deprecation warning, adds tests, and adds the 3.9 whatsnew entry. Once this PR is finalized, I think this issue can be closed.
msg355609 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-10-29 03:43
Is this whatsnew/3.8.rst entry correct and complete?

'''
The :func:`asyncio.coroutine` :term:`decorator` is deprecated and will be removed in version 3.10.  Instead of ``@asyncio.coroutine``, use :keyword:`async def` instead. (Contributed by Andrew Svetlov in :issue:`36921`.)
'''
msg355611 - (view) Author: Kyle Stanley (aeros) * (Python triager) Date: 2019-10-29 04:32
> Is this whatsnew/3.8.rst entry correct and complete?

> The :func:`asyncio.coroutine` :term:`decorator` is deprecated and will be removed in version 3.10.  Instead of ``@asyncio.coroutine``, use :keyword:`async def` instead. (Contributed by Andrew Svetlov in :issue:`36921`.)

It looks to be fully consistent with the asyncio docs: https://docs.python.org/3/library/asyncio-task.html#generator-based-coroutines. From the blame of the documentation, it seems like Yury added the deprecation notice (GH-9314) to that section with the listed release for removal as 4.0, and then changed it to 3.10 (GH-9579). At a later date, Andrew implemented the actual deprecation (GH-13346).

So as far as I can tell, yes.
History
Date User Action Args
2019-10-29 04:32:44aerossetmessages: + msg355611
2019-10-29 03:43:04rhettingersetnosy: + rhettinger
messages: + msg355609
2019-10-29 01:59:34aerossetmessages: + msg355605
stage: patch review ->
2019-10-29 01:56:30aerossetstage: patch review
pull_requests: + pull_request16503
2019-10-29 00:10:28aerossetmessages: + msg355603
stage: patch review -> (no value)
2019-10-29 00:07:13aerossetstage: patch review
pull_requests: + pull_request16502
2019-10-27 18:18:20aerossetmessages: + msg355491
2019-10-27 17:34:29asvetlovsetmessages: + msg355488
2019-10-27 17:32:23yselivanovsetmessages: + msg355487
2019-10-27 17:15:01aerossetmessages: + msg355483
2019-10-27 08:47:50asvetlovsetmessages: + msg355457
2019-10-27 07:32:13aerossetmessages: + msg355451
2019-10-27 07:25:51aerossetmessages: + msg355450
2019-10-27 07:01:58asvetlovsetmessages: + msg355448
2019-10-27 03:00:59aerossetstatus: pending -> open
nosy: + aeros
messages: + msg355443

2018-10-18 13:18:34serhiy.storchakasetstatus: open -> pending
2018-09-25 19:15:47serhiy.storchakasetmessages: + msg326397
2018-09-25 19:05:23yselivanovsetmessages: + msg326395
2018-09-25 18:57:54miss-islingtonsetnosy: + miss-islington
messages: + msg326393
2018-09-25 18:54:37yselivanovsetstatus: closed -> open
resolution: fixed ->
messages: + msg326392

stage: resolved -> (no value)
2018-09-25 18:53:46yselivanovsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-09-25 18:51:36miss-islingtonsetpull_requests: + pull_request8979
2018-09-25 18:51:24yselivanovsetmessages: + msg326390
2018-09-24 19:02:38xtreaksetnosy: + xtreak
2018-09-24 17:15:33yselivanovsetmessages: + msg326272
2018-09-24 17:06:13serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg326271
2018-09-24 16:48:28yselivanovsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8945
2018-09-24 16:42:33yselivanovcreate