This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: StreamWriter.wait_closed() can hang indefinitely.
Type: behavior Stage:
Components: asyncio Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: François Voron, aeros, asvetlov, barry, kumaraditya, lys.nikolaou, mdk, r.david.murray, r3owen, seer, tomchristie, yselivanov
Priority: normal Keywords:

Created on 2020-02-26 10:36 by tomchristie, last changed 2022-04-11 14:59 by admin.

Messages (6)
msg362688 - (view) Author: Tom Christie (tomchristie) * Date: 2020-02-26 10:36
Raising an issue that's impacting us on `httpx`.

It appears that in some cases SSL unwrapping can cause `.wait_closed()` to hang indefinately.

Trio are particularly careful to work around this case, and have an extensive comment on it: https://github.com/python-trio/trio/blob/31e2ae866ad549f1927d45ce073d4f0ea9f12419/trio/_ssl.py#L779-L829 

Originally raised via https://github.com/encode/httpx/issues/634

Tested on:

* Python 3.7.6
* Python 3.8.1

```
import asyncio
import ssl
import certifi

hostname = 'login.microsoftonline.com'
context = ssl.create_default_context()
context.load_verify_locations(cafile=certifi.where())

async def main():
    reader, writer = await asyncio.open_connection(hostname, 443, ssl=context)
    print('opened')
    writer.close()
    print('close started')
    await writer.wait_closed()
    print('close completed')

asyncio.run(main())
```
msg362738 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2020-02-26 21:02
Can reproduce in:

- 3.8.0
- 3.8.2
- 3.7.6
msg387644 - (view) Author: Russell Owen (r3owen) * Date: 2021-02-25 00:15
I am also seeing this in Python 3.8.6. I am not using SSL, but am simply calling `await writer.wait_closed()` on an `asyncio.StreamWriter`. Sometimes it works quickly and sometimes it hangs indefinitely.
msg387645 - (view) Author: Russell Owen (r3owen) * Date: 2021-02-25 00:16
Regarding my previous comment: I have never seen this in Python 3.7 (though I see that this particular bug is listed as being present there) so it may be a different underlying issue.
msg400863 - (view) Author: Artem (seer) Date: 2021-09-01 18:16
See this issue on 3.9.6
No SSL, but plain sockets.
This seems to appear when writer.write/writer.drain was cancelled, and writer.close/writer.wait_closed called after this.
msg414423 - (view) Author: Kumar Aditya (kumaraditya) * (Python triager) Date: 2022-03-03 11:14
On main branch with rewritten SSL implementation GH-31275 bpo-44011, this raises TimeoutError:

--------------------------------------------------------------------------------
opened
close started

Traceback (most recent call last):
  File "/workspaces/cpython/main.py", line 18, in <module>
    asyncio.run(main())
    ^^^^^^^^^^^^^^^^^^^
  File "/workspaces/cpython/Lib/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/cpython/Lib/asyncio/base_events.py", line 645, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/workspaces/cpython/main.py", line 14, in main
    await writer.wait_closed()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/cpython/Lib/asyncio/streams.py", line 344, in wait_closed
    await self._protocol._get_close_waiter(self)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: SSL shutdown timed out
--------------------------------------------------------------------------------

This issue has been fixed, it can be closed @asvetlov.
History
Date User Action Args
2022-04-11 14:59:27adminsetgithub: 83939
2022-03-03 11:14:26kumaradityasetversions: - Python 3.8, Python 3.9, Python 3.10
nosy: + kumaraditya

messages: + msg414423

type: performance -> behavior
2021-09-01 21:51:43pablogsalsetnosy: - pablogsal
components: - Build, Tests, email, C API, Parser
2021-09-01 21:47:12beblostanislavsetnosy: + barry, pablogsal, lys.nikolaou, r.david.murray

type: behavior -> performance
components: + Build, Tests, email, C API, Parser
versions: + Python 3.10, Python 3.11, - Python 3.7
2021-09-01 18:16:39seersetnosy: + aeros, seer

messages: + msg400863
versions: + Python 3.9
2021-02-25 00:16:14r3owensetmessages: + msg387645
2021-02-25 00:15:11r3owensetnosy: + r3owen
messages: + msg387644
2020-02-26 21:02:09mdksetnosy: + mdk
messages: + msg362738
2020-02-26 10:53:32François Voronsetnosy: + François Voron

title: StreamWriter.wait_closed() can hang indefinately. -> StreamWriter.wait_closed() can hang indefinitely.
2020-02-26 10:36:06tomchristiecreate