An overhaul of asyncio documentation is long overdue.  Here's the structure for it that I have in mind:

- Introduction
(what is asyncio and async/await)

- A quick tutorial
(show how to use asyncio.run() and basic functions like asyncio.sleep() and teach that asyncio programs are all about async/await and *not* about callbacks or event loops)

- High-level APIs
(Tasks, Streams, Subprocesses, few other functions)

- Low-level APIs
  - Preface (talk a bit about everything: what's an event loop, what is a Future and a Transport)
  - Futures
  - Event loop APIs
  - Transports and Protocols (when to use and when not to use them)

- Tutorials
  - High-level networking server
  - HTTP application
  - Low-level protocol implementation using Transports
  - etc

The plan for updating things looks good, and I think the ordering of
tasks is good -- rearranging the APIs is a pretty simple first step,
and it matches other chapters in the Library Reference, which list
more important modules first and more specialized obscure modules
later.  Then we can work on writing the additional material.

Yury: do you want to do the re-organization into high- and low-level APIs?  If you'd like me to do it, I'll need a more explicit listing of which APIs fall into which category.  Also, do we want to create an explicit high-level and low-level sections and push everything else down a level, or just leave it implicit in the ordering?

> Yury: do you want to do the re-organization into high- and low-level APIs?  If you'd like me to do it, I'll need a more explicit listing of which APIs fall into which category.

Sure, I'll update this issue tomorrow or the day after with a more detailed plan.  Or maybe I'll start working on the 1st PR and will just give you push privileges to my repo.  Would be cool if we can push the first rewrite live as soon as possible.

> Also, do we want to create an explicit high-level and low-level sections and push everything else down a level, or just leave it implicit in the ordering?

Yes, I'd go for 2 explicitly separate sections.  I think it's important to focus average readers' attention at the high-level bits and pieces, leaving low-level APIs and details to framework/library authors.

And, again, thanks for help.

Adding Carol to the nosy list as she seems to be interested in this topic too.

I'm a developer using Python in my application. I just spent the last couple of weeks learning about asyncio with the present documentation. I am very happy to see that work is underway for improved documentation. 

I would be glad to take on writing tasks or to review drafts, if there is a way to plug myself into that work.

I like the general structure. 

Some specific topics which would have helped me in my recent learning:

* As an application developer using an event loop as part of the application, how do I develop a Protocol specific to my app, and use with Streams?  An important part of this is being clear what the interface is between Transport and Protocol. This interface resides in both classes; Protocol calls Transport methods, and Transport calls Protocol methods.

* As an application developer using an event loop as part of the application, how do I develop a Transport specific to my app, and use it with Streams?  (In my case, I have two transports in mind: a file, containing data archived from a TCP network port, and an RS232 serial connection, carrying data that might otherwise be sent via a TCP network port. It seems to me that I should be able at run-time to select from any of those three Transports, and apply my app-specific Protocol when creating my connection.)

* As an application developer, how do I give a function which is not a coroutine to the event loop for scheduling and execution?  (I think the answer is loop.call_soon(), but the docs don't say "to run your function call loop.call_soon(myfunc...)", they say "call_soon(): Arrange for a callback to be called as soon as possible". Not the same thing. And, is a "callback" different "an arbitrary function"? Not clear.)

* As an application developer, how do I make an syncio-based streaming server and streaming client respond cleanly to interrupts, e.g. control-C in a command-line app?  (Existing docs hints at catching signals, but don't show how to combine those with exception handlers in the coroutine to shut down connections cleanly.)

* As a Transport developer, what interface must I provide between Transport and Protocol? What services does the EventLoop provide to help implement my Transport? As mentioned before, to me it looks like the Transport-Protocol interface resides in both classes; Protocol calls Transport methods, and Transport calls Protocol methods.  The EventLoop services useful to Transports are not gathered in a discussion of Transports. There is only an enumeration of the EventLoop methods, regardless of purpose.

Editorial checks I suggest making to the revision:

Rewrite to describe the behaviour in the doc, instead of delegating to a PEP. No more "this class is almost like the thing in PEP xyz, except for these differences". Describe the asyncio class, and steal text from PEP xyz as needed to do that.

Every section about a thing must start by saying what that thing is and what it is used for.  There are some big counterexamples in the current text. 

e.g. "19.5.1.2. Calls" <https://docs.python.org/3/library/asyncio-eventloop.html#calls> starts out with: "Most asyncio functions don’t accept keywords. If you want to pass keywords to your callback, use functools.partial()."  That does not describe what "calls" are, nor what they are used for. And, in this case, a better title might be, "Getting the event loop to call regular functions".

e.g. "19.5.1.4. Futures" <https://docs.python.org/3/library/asyncio-eventloop.html#futures> has no starting text. This is the first heading mentioning Futures when reading sequentially in the asyncio doc, so I interpreted it as a place to learn about Futures. It is not. This section would be better titles "Eventloop methods for Futures". The proposed "Low-level APIs" section might solve this problem by explaining the nature and purpose of Futures before mentioning the EventLoop factory for Futures objects.

Consider separating the explanation of the nature and purpose of a thing from the reference to the methods of the thing. The proposed structure talks about "High-Level APIs" and "Low-Level APIs", but not "Architecture" and "API Reference". I would suggest injecting an "Architecture" section which can give the nature and purpose of each of the public classes in Asyncio, without enumerating their methods. Then the sections labelled "High-Level APIs" and "Low-Level APIs" can be the API reference.

I hope these suggestions are helpful. I'll be monitoring this issue to see how I can help.

Status update: I'm working on a first rewrite. I expect to have something to review on a couple of weeks.

New changeset 7c7605ff1133cf757cac428c483827f666c7c827 by Yury Selivanov in branch 'master':
bpo-33649: First asyncio docs improvement pass (GH-9142)
https://github.com/python/cpython/commit/7c7605ff1133cf757cac428c483827f666c7c827

The first PR has been just landed.

The plan is to work in 'master' branch and when the docs are ready we backport them to 3.7 in one big PR.

So far the following files were updated/reviewed:

     Doc/library/asyncio-eventloop.rst
     Doc/library/asyncio-eventloops.rst -> Doc/library/asyncio-policy.rst
     Doc/library/asyncio-exceptions.rst
     Doc/library/asyncio-platforms.rst 
     Doc/library/asyncio-policy.rst    
     Doc/library/asyncio-protocol.rst  
     Doc/library/asyncio-queue.rst     
     Doc/library/asyncio-stream.rst    
     Doc/library/asyncio-subprocess.rst 

Please feel free to submit PR to fix/improve the above (but not other files as I'm going to be making some heavy edit passes on them).

New changeset 8be876e44b2dffb96b551029454cbfb16d697992 by Yury Selivanov in branch 'master':
bpo-33649: Cleanup asyncio/streams and asyncio/synchronization docs (GH-9192)
https://github.com/python/cpython/commit/8be876e44b2dffb96b551029454cbfb16d697992

New changeset aca819fb494d4801b3e5b5b507b17cab772c1b40 by Yury Selivanov (Bumsik Kim) in branch 'master':
bpo-33649: Fix doc to reflect changes in 47cd10d (or bpo-23347) (GH-9219)
https://github.com/python/cpython/commit/aca819fb494d4801b3e5b5b507b17cab772c1b40

Hi, I came from #33986. I noticed that the new doc still does not reflect a design change on SubprocessTransport.close() done in #23347. I made a PR to fix that.

BTW this is opposed to the original PEP 3156: https://www.python.org/dev/peps/pep-3156/#subprocess-transports . I don't know if it is commonly happens.

New changeset 5b7cbd602e57265604d6c099fd174b1c7917f861 by Yury Selivanov (Carol Willing) in branch 'master':
bpo-33649: Edit asyncio eventloop doc - second pass (GH-9233)
https://github.com/python/cpython/commit/5b7cbd602e57265604d6c099fd174b1c7917f861

New changeset 4e824e96491f33c8a8462aa4970c55942064ae76 by Miss Islington (bot) (Carol Willing) in branch 'master':
bpo-33649: Polish asyncio subprocess and sync docs (GH-9285)
https://github.com/python/cpython/commit/4e824e96491f33c8a8462aa4970c55942064ae76

New changeset c9d66f0ed4f07b9d184d22abbfdd4c3c8e2702df by Carol Willing in branch 'master':
bpo-33649 Polish asyncio docs on queues, protocols, and subproccesses (#9306)
https://github.com/python/cpython/commit/c9d66f0ed4f07b9d184d22abbfdd4c3c8e2702df

New changeset 3faaa8857a42a36383bb18425444e597fc876797 by Carol Willing (Yury Selivanov) in branch 'master':
bpo-33649: Refresh Tasks and Futures pages (#9314)
https://github.com/python/cpython/commit/3faaa8857a42a36383bb18425444e597fc876797

New changeset 6c7316439d966cdbc76ffcc005e5d9c563064ba2 by Yury Selivanov in branch 'master':
bpo-33649: Refresh asyncio docs landing page (GH-9322)
https://github.com/python/cpython/commit/6c7316439d966cdbc76ffcc005e5d9c563064ba2

New changeset 7372c3bbefb4763dbd1b6d66f7971bef28c0f056 by Yury Selivanov in branch 'master':
bpo-33649: Add high-level APIs cheat-sheet (GH-9319)
https://github.com/python/cpython/commit/7372c3bbefb4763dbd1b6d66f7971bef28c0f056

New changeset 805e27eff65d51f7aea2c00ccbb4f5d44f4499f2 by Yury Selivanov in branch 'master':
bpo-33649: Fix asyncio-dev (GH-9324)
https://github.com/python/cpython/commit/805e27eff65d51f7aea2c00ccbb4f5d44f4499f2

New changeset 5cc583d94095ed3bb543fa2f032f9593a6315a52 by Yury Selivanov (Bumsik Kim) in branch 'master':
bpo-33649: Clarify protocol_factory as a method parameter (GH-9330)
https://github.com/python/cpython/commit/5cc583d94095ed3bb543fa2f032f9593a6315a52

New changeset 394374e30c85f6eacddbbfc7471aab62b54ce021 by Yury Selivanov in branch 'master':
bpo-33649: Add low-level APIs index. (GH-9364)
https://github.com/python/cpython/commit/394374e30c85f6eacddbbfc7471aab62b54ce021

New changeset 3085534c398e6b181e7a9ac0cb9c80f3c670f2b9 by Yury Selivanov in branch 'master':
bpo-33649: Add a hello world example to asyncio.rst (GH-9374)
https://github.com/python/cpython/commit/3085534c398e6b181e7a9ac0cb9c80f3c670f2b9

New changeset 1fa2ec49bec50bea1847b558b883c5c904334734 by Yury Selivanov (Elvis Pranskevichus) in branch 'master':
bpo-33649: A copy-editing pass on asyncio documentation (GH-9376)
https://github.com/python/cpython/commit/1fa2ec49bec50bea1847b558b883c5c904334734

New changeset 512d7101098b971837cbb406942215244f636547 by Yury Selivanov in branch '3.7':
bpo-33649: Backport asyncio docs from 'master' to 3.7 (GH-9377)
https://github.com/python/cpython/commit/512d7101098b971837cbb406942215244f636547

Wow, we did this!  I've just pushed a backport of the new documentation to 3.7.

Thanks to Carol, Elvis, and Andrew for the help!

I think this issue can now be closed, we'll open new ones for:

1. A high-level intro/tutorial for asyncio.

2. A low-level tutorial (say we implement a micro-library for Redis using only low-level APIs)

3. An asyncio "Architecture" section as suggested by Jim DeLaHunt.

Ned, do we auto run docs build for Python 3.7?  Or do we only build them on release?

> Ned, do we auto run docs build for Python 3.7?  Or do we only build them on release?

Never mind, the docs for 3.7 have been updated!

>> Ned, do we auto run docs build for Python 3.7?  Or do we only build them on release?

The online html docs at https://docs.python.org/, like https://docs.python.org/3.7/, are rebuilt every 3 hours from the current branch heads for feature and maintenance releases.  We also make available archive copies of the docs for each release here:
https://www.python.org/doc/versions/

>Never mind, the docs for 3.7 have been updated!

What's up, Doc?

New changeset ac94e38d076aebc56c2ff96a249b5e40f32633ea by Yury Selivanov in branch 'master':
bpo-33649: Note that asyncio.run() calls shutdown_asyncgens() (GH-9380)
https://github.com/python/cpython/commit/ac94e38d076aebc56c2ff96a249b5e40f32633ea

New changeset 1f4ea580675d7b30d3906d78c500997d7d96995d by Miss Islington (bot) in branch '3.7':
bpo-33649: Note that asyncio.run() calls shutdown_asyncgens() (GH-9380)
https://github.com/python/cpython/commit/1f4ea580675d7b30d3906d78c500997d7d96995d

New changeset b042cf10c6084d14279c55a7e0d2d7595ff4e694 by Yury Selivanov in branch 'master':
bpo-33649: Fix markup; add another note that asyncio.run is 3.7+ (GH-9389)
https://github.com/python/cpython/commit/b042cf10c6084d14279c55a7e0d2d7595ff4e694

New changeset 45452b738b7f94221a94e903fb5975222fbb7a8f by Miss Islington (bot) in branch '3.7':
bpo-33649: Fix markup; add another note that asyncio.run is 3.7+ (GH-9389)
https://github.com/python/cpython/commit/45452b738b7f94221a94e903fb5975222fbb7a8f

New changeset 471503954a91d86cf04228c38134108c67a263b0 by Yury Selivanov in branch 'master':
bpo-33649: Add a high-level section about Futures; few quick fixes (GH-9403)
https://github.com/python/cpython/commit/471503954a91d86cf04228c38134108c67a263b0

New changeset 73c0006e71683b7d5b28192f18a2b9796e4195ef by Miss Islington (bot) in branch '3.7':
bpo-33649: Add a high-level section about Futures; few quick fixes (GH-9403)
https://github.com/python/cpython/commit/73c0006e71683b7d5b28192f18a2b9796e4195ef

New changeset e247b46cba4f4d32ea96a15dbc36d73265171106 by Yury Selivanov in branch 'master':
bpo-33649: More improvements (GH-9439)
https://github.com/python/cpython/commit/e247b46cba4f4d32ea96a15dbc36d73265171106

New changeset 8e5ef58c10a1154f824d5875c2d89794a800eadc by Miss Islington (bot) in branch '3.7':
bpo-33649: More improvements (GH-9439)
https://github.com/python/cpython/commit/8e5ef58c10a1154f824d5875c2d89794a800eadc

New changeset db1a80e97aa7217c561fb3627f70be1882de9534 by Yury Selivanov in branch 'master':
bpo-33649: Fix gather() docs; fix title; few other nits. (GH-9475)
https://github.com/python/cpython/commit/db1a80e97aa7217c561fb3627f70be1882de9534

New changeset e45662c28bfc84aa3674463a2995e45da4d63793 by Yury Selivanov (Miss Islington (bot)) in branch '3.7':
bpo-33649: Fix gather() docs; fix title; few other nits. (GH-9475) (GH-9481)
https://github.com/python/cpython/commit/e45662c28bfc84aa3674463a2995e45da4d63793

New changeset 9d74658f0a6e8a9b8d6dcf199dda886f35c6ad68 by Joel Rosdahl in branch 'master':
bpo-40499: Mention that asyncio.wait() needs a non-empty aws set (GH-19900)
https://github.com/python/cpython/commit/9d74658f0a6e8a9b8d6dcf199dda886f35c6ad68