classification
Title: Deprecate explicit loop parameter in all public asyncio APIs
Type: enhancement Stage: resolved
Components: asyncio, Documentation Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: asvetlov, docs@python, dtrauma, eamanu, mbussonn, miss-islington, xtreak, yselivanov
Priority: normal Keywords: patch

Created on 2019-03-19 19:55 by dtrauma, last changed 2019-09-12 14:22 by asvetlov. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 13670 closed eamanu, 2019-05-30 00:50
PR 13671 merged eamanu, 2019-05-30 01:43
PR 13833 merged miss-islington, 2019-06-05 05:46
PR 13920 merged eamanu, 2019-06-08 17:49
PR 13950 merged eamanu, 2019-06-10 23:39
PR 15835 merged miss-islington, 2019-09-10 10:55
PR 15841 merged miss-islington, 2019-09-10 11:46
PR 15889 merged asvetlov, 2019-09-10 16:44
PR 15896 closed miss-islington, 2019-09-11 08:20
PR 15901 merged asvetlov, 2019-09-11 09:34
PR 16033 merged asvetlov, 2019-09-12 11:44
PR 16039 merged miss-islington, 2019-09-12 12:40
Messages (29)
msg338406 - (view) Author: (dtrauma) Date: 2019-03-19 19:55
https://docs.python.org/3.7/library/asyncio-task.html#running-tasks-concurrently

For asyncio.gather, the docs should probably say

The loop argument is deprecated and scheduled for removal in Python 3.10.

as they do for all the other loop arguments of other functions.
msg338452 - (view) Author: Emmanuel Arias (eamanu) * Date: 2019-03-20 10:15
Hi,

Reading the asyncio.gather code not seem to be deprecrated.
loop is used on a lot of line of code. 

Maybe, the deprecate idea is on other place where I don't know.
msg338984 - (view) Author: (dtrauma) Date: 2019-03-27 20:41
Just to be clear, I don't know if loop is deprecated on this function like on all the others, I just suspect it to be. But it currently is completely undocumented, which either way is a bug. :)
msg338985 - (view) Author: Emmanuel Arias (eamanu) * Date: 2019-03-27 20:52
@dtrauma jus for clarify. You say that if loop is not deprecated document it else document it.


Right?
msg338986 - (view) Author: (dtrauma) Date: 2019-03-27 20:54
Yes, exactly, document deprecation status XOR what it does :)
msg343809 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-28 17:50
*Passing* loop to gather should be deprecated.

*Using* loop by *internal logic* is pretty fine, we do it in asyncio everywhere.

Yuri, I think we should deprecate explicit loop everywhere in non-deprecated asyncio API by Python 3.8.

We can do it even in Python beta I think.
msg343810 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-05-28 17:52
> Yuri, I think we should deprecate explicit loop everywhere in non-deprecated asyncio API by Python 3.8.

+1
msg343882 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-29 12:14
A champion is welcome :)
msg343887 - (view) Author: Emmanuel Arias (eamanu) * Date: 2019-05-29 14:43
hello, I will work on it, if there are no objection. :-)
msg343888 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-29 14:47
Thank you.

The change is pretty straightforward.

There is no need to jumbo PR, you can split the work into PR-per-module if it is more comfortable to you.

I'm ok with reviewing any approach.
msg343889 - (view) Author: Emmanuel Arias (eamanu) * Date: 2019-05-29 14:52
Hi
> There is no need to jumbo PR, you can split the work into PR-per-module if it is more comfortable to you.
Yes, I think that is better split I will try it.
> I'm ok with reviewing any approach.
Thanks!
msg344673 - (view) Author: miss-islington (miss-islington) Date: 2019-06-05 05:45
New changeset 6d64a8f49eb321116f585c4b036c81bb976d2d5c by Miss Islington (bot) (Emmanuel Arias) in branch 'master':
bpo-36373:  Deprecate explicit loop parameter in all public asyncio APIs [streams] (GH-13671)
https://github.com/python/cpython/commit/6d64a8f49eb321116f585c4b036c81bb976d2d5c
msg344674 - (view) Author: miss-islington (miss-islington) Date: 2019-06-05 06:01
New changeset 8899b11b95f08e2e03478f2acad336ad5933a2d1 by Miss Islington (bot) in branch '3.8':
bpo-36373:  Deprecate explicit loop parameter in all public asyncio APIs [streams] (GH-13671)
https://github.com/python/cpython/commit/8899b11b95f08e2e03478f2acad336ad5933a2d1
msg351618 - (view) Author: miss-islington (miss-islington) Date: 2019-09-10 10:55
New changeset 537877d85d1c27d2c2f5189e39da64a7a0c413d3 by Miss Islington (bot) (Emmanuel Arias) in branch 'master':
 bpo-36373: Deprecate explicit loop parameter in all public asyncio APIs [locks] (GH-13920)
https://github.com/python/cpython/commit/537877d85d1c27d2c2f5189e39da64a7a0c413d3
msg351624 - (view) Author: miss-islington (miss-islington) Date: 2019-09-10 11:26
New changeset bb8fc8bd309419c159b632068dff73c3c76d478c by Miss Islington (bot) in branch '3.8':
bpo-36373: Deprecate explicit loop parameter in all public asyncio APIs [locks] (GH-13920)
https://github.com/python/cpython/commit/bb8fc8bd309419c159b632068dff73c3c76d478c
msg351626 - (view) Author: miss-islington (miss-islington) Date: 2019-09-10 11:46
New changeset 9008be303a89bfab8c3314c6a42330b5523adc8b by Miss Islington (bot) (Emmanuel Arias) in branch 'master':
bpo-36373: Deprecate explicit loop parameter in all public asyncio APIs [queue] (GH-13950)
https://github.com/python/cpython/commit/9008be303a89bfab8c3314c6a42330b5523adc8b
msg351630 - (view) Author: miss-islington (miss-islington) Date: 2019-09-10 12:50
New changeset 55daf1a56163e0673654712bb92ce47a9b6f7be3 by Miss Islington (bot) in branch '3.8':
bpo-36373: Deprecate explicit loop parameter in all public asyncio APIs [queue] (GH-13950)
https://github.com/python/cpython/commit/55daf1a56163e0673654712bb92ce47a9b6f7be3
msg351651 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-09-10 13:32
async_case passes loop parameter to the queue used to collect tests and this causes below DeprecationWarning with last change over deprecation in queues and locks API.

./python.exe -m unittest Lib.unittest.test.test_async_case
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/async_case.py:116: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.
  self._asyncioCallsQueue = asyncio.Queue(loop=loop)
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/queues.py:48: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.
  self._finished = locks.Event(loop=self._loop)
.......
----------------------------------------------------------------------
Ran 7 tests in 0.083s

OK
msg351658 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-09-10 13:43
Few more tests and call sites

./python.exe -Wall -m test test_asyncio
Run tests sequentially
0:00:00 load avg: 2.41 [1/1] test_asyncio
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/locks.py:335: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.
  lock = Lock(loop=self._loop)
test_asyncio passed in 1 min 38 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1 min 38 sec
Tests result: SUCCESS

call sites 

>>> list(asyncio.tasks.as_completed([asyncio.ensure_future(asyncio.sleep(1))]))
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/tasks.py:579: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.
  done = Queue(loop=loop)
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/asyncio/queues.py:48: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.
  self._finished = locks.Event(loop=self._loop)
msg351665 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-09-10 14:06
Thanks for the report xtreak, I'll make a fix.
msg351677 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-09-10 14:33
Is it okay to have _asyncio_internal keyword with default value as False and the places where it's used internally in stdlib like AsyncMock (Condition), async_case (Queue) and other tests we can pass in True? I have a patch if that would be a good approach :)
msg351679 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-09-10 14:41
Better to avoid _asyncio_internal if not strictly necessary.
Currently, _asyncio_internal is used for protection of asyncio class direct instantiation.

E.g. stream = asyncio.Stream() is forbidden, people should use factories like stream = await asyncio.connect(...) for it.
msg351684 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-09-10 14:46
Ah okay. There were places like Condition's constructor that has a deprecation warning and we construct a Lock with loop passed in the constructor and I passed along _asyncio_internal. I would be happy to look into your PR since mine is just passing along _asyncio_internal=True for all the stdlib calls.

I think it would be good if tests ran with warnings (at least DeprecationWarning and some class of warnings) as error but not sure why :(
msg351729 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-09-10 17:05
(copying my GitHub message on this topic from https://github.com/python/cpython/pull/15889#issuecomment-530030002)


I thought we were going to be more subtle about this.

We have two scenarios:

1. Somebody creates an instance of asyncio.Lock outside of a coroutine, also manually creating the loop, etc. In this case the user has to pass the lock argument.

2. Somebody creates an instance of asyncio.Lock in a coroutine (that is, wheh asyncio.get_running_loop() returns a loop). In this case passing the loop argument is an error.

The (1) approach is how people were used to writing asyncio programs before asyncio.run() (that was the only way actually).

The (2) approach is how we want people to write asyncio programs.

There's a subtle difference between things like asyncio.gather() and asyncio.Lock. Passing loop to the former is just nonsensical. Passing loop to the latter can be a valid thing, it's the (1).

If we remove the loop parameter entirely from classes like asyncio.Lock we're making (1) impossible. I'm not sure that is what we are ready to do now.

@asvetlov thoughts?
msg351739 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-09-10 19:30
Answered in https://github.com/python/cpython/pull/15889
msg351770 - (view) Author: miss-islington (miss-islington) Date: 2019-09-11 08:20
New changeset 7264e92b718d307cc499b2f10eab7644b00f0499 by Miss Islington (bot) (Andrew Svetlov) in branch 'master':
bpo-36373: Fix deprecation warnings (GH-15889)
https://github.com/python/cpython/commit/7264e92b718d307cc499b2f10eab7644b00f0499
msg351805 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-09-11 10:40
New changeset 4601f7a49fe8ed00c4b6b70b0eda2b3922568e9b by Andrew Svetlov in branch '3.8':
[3.8] bpo-36373: Fix deprecation warnings (GH-15889) (GH-15901)
https://github.com/python/cpython/commit/4601f7a49fe8ed00c4b6b70b0eda2b3922568e9b
msg352145 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-09-12 12:40
New changeset a488879cbaf4b8b52699cadccf73bb4c271bcb29 by Andrew Svetlov in branch 'master':
bpo-36373: Deprecate explicit loop in task and subprocess API (GH-16033)
https://github.com/python/cpython/commit/a488879cbaf4b8b52699cadccf73bb4c271bcb29
msg352150 - (view) Author: miss-islington (miss-islington) Date: 2019-09-12 12:59
New changeset 345bfc990f5f3e873774051d43136b06bfed82cb by Miss Islington (bot) in branch '3.8':
bpo-36373: Deprecate explicit loop in task and subprocess API (GH-16033)
https://github.com/python/cpython/commit/345bfc990f5f3e873774051d43136b06bfed82cb
History
Date User Action Args
2019-09-12 14:22:26asvetlovsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-09-12 12:59:53miss-islingtonsetmessages: + msg352150
2019-09-12 12:40:55miss-islingtonsetpull_requests: + pull_request15661
2019-09-12 12:40:44asvetlovsetmessages: + msg352145
2019-09-12 11:44:58asvetlovsetpull_requests: + pull_request15655
2019-09-11 10:40:39asvetlovsetmessages: + msg351805
2019-09-11 09:34:36asvetlovsetpull_requests: + pull_request15542
2019-09-11 08:20:54miss-islingtonsetpull_requests: + pull_request15537
2019-09-11 08:20:27miss-islingtonsetmessages: + msg351770
2019-09-10 19:30:59asvetlovsetmessages: + msg351739
2019-09-10 17:05:26yselivanovsetmessages: + msg351729
2019-09-10 17:05:10yselivanovsetmessages: - msg351728
2019-09-10 17:04:39yselivanovsetmessages: + msg351728
2019-09-10 16:44:20asvetlovsetpull_requests: + pull_request15530
2019-09-10 14:46:43xtreaksetmessages: + msg351684
2019-09-10 14:41:44asvetlovsetmessages: + msg351679
2019-09-10 14:33:19xtreaksetmessages: + msg351677
2019-09-10 14:06:34asvetlovsetmessages: + msg351665
2019-09-10 13:43:40xtreaksetmessages: + msg351658
2019-09-10 13:32:39xtreaksetnosy: + xtreak
messages: + msg351651
2019-09-10 12:50:17miss-islingtonsetmessages: + msg351630
2019-09-10 11:46:23miss-islingtonsetpull_requests: + pull_request15488
2019-09-10 11:46:15miss-islingtonsetmessages: + msg351626
2019-09-10 11:26:57miss-islingtonsetmessages: + msg351624
2019-09-10 10:55:45miss-islingtonsetpull_requests: + pull_request15480
2019-09-10 10:55:10miss-islingtonsetmessages: + msg351618
2019-06-10 23:39:36eamanusetpull_requests: + pull_request13818
2019-06-08 17:49:25eamanusetpull_requests: + pull_request13793
2019-06-05 06:01:04miss-islingtonsetmessages: + msg344674
2019-06-05 05:46:20miss-islingtonsetpull_requests: + pull_request13712
2019-06-05 05:45:57miss-islingtonsetnosy: + miss-islington
messages: + msg344673
2019-05-30 01:43:11eamanusetpull_requests: + pull_request13560
2019-05-30 00:50:45eamanusetkeywords: + patch
stage: patch review
pull_requests: + pull_request13559
2019-05-29 14:55:09asvetlovsettitle: asyncio.gather: no docs for deprecated loop parameter -> Deprecate explicit loop parameter in all public asyncio APIs
2019-05-29 14:52:48eamanusetmessages: + msg343889
title: Deprecate explicit loop parameter in all public asyncio APIs -> asyncio.gather: no docs for deprecated loop parameter
2019-05-29 14:50:49asvetlovsetversions: + Python 3.8, - Python 3.7
2019-05-29 14:50:43asvetlovsettitle: asyncio.gather: no docs for deprecated loop parameter -> Deprecate explicit loop parameter in all public asyncio APIs
2019-05-29 14:47:52asvetlovsetmessages: + msg343888
2019-05-29 14:43:34eamanusetmessages: + msg343887
2019-05-29 12:14:34asvetlovsetmessages: + msg343882
2019-05-28 23:47:37mbussonnsetnosy: + mbussonn
2019-05-28 17:52:43yselivanovsetmessages: + msg343810
2019-05-28 17:50:25asvetlovsetmessages: + msg343809
2019-03-27 20:54:16dtraumasetmessages: + msg338986
2019-03-27 20:52:44eamanusetmessages: + msg338985
2019-03-27 20:41:06dtraumasetmessages: + msg338984
2019-03-20 10:15:50eamanusetnosy: + eamanu
messages: + msg338452
2019-03-20 02:47:58xtreaksetnosy: + asvetlov, yselivanov
components: + asyncio
2019-03-19 19:55:35dtraumacreate