This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Asyncio SSL keep-alive connections raise errors after loop close.
Type: behavior Stage: resolved
Components: asyncio, SSL Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, christian.heimes, kumaraditya, tomchristie, yselivanov
Priority: normal Keywords:

Created on 2019-04-24 08:48 by tomchristie, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (14)
msg340764 - (view) Author: Tom Christie (tomchristie) * Date: 2019-04-24 08:48
If an asyncio SSL connection is left open (eg. any kind of keep-alive connection) then after closing the event loop, an exception will be raised...

Python:

```
import asyncio
import ssl
import certifi


async def f():
    ssl_context = ssl.create_default_context()
    ssl_context.load_verify_locations(cafile=certifi.where())
    await asyncio.open_connection('example.org', 443, ssl=ssl_context)


loop = asyncio.get_event_loop()
loop.run_until_complete(f())
loop.close()
```

Traceback:

```
$ python example.py 
Fatal write error on socket transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x10e7874a8>
transport: <_SelectorSocketTransport fd=8>
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 868, in write
    n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor
Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x10e7874a8>
transport: <_SelectorSocketTransport closing fd=8>
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 868, in write
    n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py", line 676, in _process_write_backlog
    self._transport.write(chunk)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 872, in write
    self._fatal_error(exc, 'Fatal write error on socket transport')
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 681, in _fatal_error
    self._force_close(exc)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 693, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 677, in call_soon
    self._check_closed()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 469, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
```

It looks to me like the original "OSError: [Errno 9] Bad file descriptor" probably shouldn't be raised in any case - if when attempting to tear down the SSL connection, then we should probably pass silently in the case that the socket has already been closed uncleanly.

Bought to my attention via: https://github.com/encode/httpcore/issues/16
msg340765 - (view) Author: Tom Christie (tomchristie) * Date: 2019-04-24 09:10
This appears somewhat related: https://bugs.python.org/issue34506

As it *also* logs exceptions occuring during `_fatal_error` and `_force_close`.
msg343776 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-28 13:11
From my understanding, the correct code should close all transports and wait for their connection_lost() callbacks before closing the loop.
msg343780 - (view) Author: Tom Christie (tomchristie) * Date: 2019-05-28 14:04
> From my understanding, the correct code should close all transports and wait for their connection_lost() callbacks before closing the loop.

Ideally, yes, although we should be able to expect that an SSL connection that hasn't been gracefully closed wouldn't loudly error on teardown like that.

In standard sync code, the equivelent would running something like this...

```python
session = requests.Session()
session.get('https://example.com/')
```

We wouldn't expect a traceback to be raised on exiting. (Even though the user *hasn't* explicitly closed the session, and even though a keep alive SSL connection will be open at the point of exit.)
msg343883 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-29 13:13
I would say that if requests a designed from scratch more idiomatic way could be

with requests.Session() as session:
    session.get('https://example.com/')

or

session = requests.Session()
session.get('https://example.com/')
session.close()

Like the recommended way to handle files.
msg343977 - (view) Author: Tom Christie (tomchristie) * Date: 2019-05-30 14:00
Right, and `requests` *does* provide both those styles.

The point more being that *not* having closed the transport at the point of exit shouldn't end up raising a hard error. It doesn't raise errors in sync-land, and it shouldn't do so in async-land.

Similarly, we wouldn't expect an open file resource to cause errors to be raised at the point of exit.
msg343978 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-30 14:10
The difference is that socket.close() is an instant call.
After socket.close() the socket is done.
But transport.close() doesn't close the transport instantly.
asyncio requires at least one loop iteration for calling protocol.connection_lost() and actual socket closing.

In case of SSL it may take much longer.

Sorry, that's how asyncio is designed.
msg343979 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-05-30 14:42
> Sorry, that's how asyncio is designed.

Andrew, couldn't we provide a "stream.terminate()" method (not a coroutine) that would do the following:

* close the transport
* set a flag in the protocol that the stream has been terminated.  When the flag is set, the protocol simply ignores all errors (i.e. they are never shown to the user or logged)

This way Tom could have a weakref to the stream object from his high-level wrapper, and whenever the wrapper object is dereferenced it could terminate its stream.

> Sorry, that's how asyncio is designed.

I think it's a real problem, let's try to find out if we can provide a solution.
msg343980 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-30 14:52
It's not about streams only.
The stream protocol can have such flag, sure.

But transport emits a warning like "unclosed transport ..."

Not sure if we have to drop this warning, it enforces writing good code that controls all created resources lifecycle.
msg343981 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2019-05-30 14:57
> Not sure if we have to drop this warning, it enforces writing good code that controls all created resources lifecycle.

Right, maybe we add "transport.terminate()" as well?  Synchronously & immediately closing a transport is a valid use case. 

TBH I don't see why we absolutely must wait the "connection_lost" callback call.  I mean it would be reasonable to allow users to terminate the connection (even if it means that in some cases, like SSL, it won't be correctly closed) and not care about what happens to it next.
msg343984 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-30 15:54
Sorry, I'm not comfortable with such change just before the feature freeze.
The idea is maybe good but let's discuss and implement it later.
msg379222 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-10-21 16:46
This seems to be an asyncio problem.
msg414425 - (view) Author: Kumar Aditya (kumaraditya) * (Python triager) Date: 2022-03-03 11:22
The original issue is fixed on main branch with bpo-44011 GH-31275,
It now only raises warnings but no exceptions:

-----------------------------------------------------------
(env) @kumaraditya303 ➜ /workspaces/cpython (latin1 ✗) $ python main.py
/workspaces/cpython/main.py:12: DeprecationWarning: There is no current event loop
  loop = asyncio.get_event_loop()
/workspaces/cpython/env/lib/python3.11/site-packages/certifi/core.py:36: DeprecationWarning: path is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  _CACERT_CTX = get_path("certifi", "cacert.pem")
sys:1: ResourceWarning: unclosed <socket.socket fd=7, family=2, type=1, proto=6, laddr=('172.16.5.4', 49202), raddr=('93.184.216.34', 443)>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspaces/cpython/Lib/asyncio/sslproto.py:116: ResourceWarning: unclosed transport <asyncio._SSLProtocolTransport object>
  _warnings.warn(
/workspaces/cpython/Lib/asyncio/selector_events.py:710: ResourceWarning: unclosed transport <_SelectorSocketTransport fd=7>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

-----------------------------------------------------------

@asvetlvo This can be closed now.
msg414825 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-10 02:07
Thanks!
History
Date User Action Args
2022-04-11 14:59:14adminsetgithub: 80890
2022-03-10 02:07:51asvetlovsetstatus: open -> closed
resolution: fixed
messages: + msg414825

stage: resolved
2022-03-03 11:22:48kumaradityasetversions: + Python 3.11, - Python 3.7
nosy: + kumaraditya

messages: + msg414425

type: behavior
2020-10-21 16:46:37christian.heimessetassignee: christian.heimes ->
messages: + msg379222
nosy: christian.heimes, asvetlov, tomchristie, yselivanov
2019-05-30 15:54:49asvetlovsetmessages: + msg343984
2019-05-30 14:57:32yselivanovsetmessages: + msg343981
2019-05-30 14:52:05asvetlovsetmessages: + msg343980
2019-05-30 14:42:53yselivanovsetmessages: + msg343979
2019-05-30 14:10:43asvetlovsetmessages: + msg343978
2019-05-30 14:00:41tomchristiesetmessages: + msg343977
2019-05-29 13:13:55asvetlovsetmessages: + msg343883
2019-05-28 14:04:19tomchristiesetmessages: + msg343780
2019-05-28 13:11:30asvetlovsetmessages: + msg343776
2019-04-24 09:10:09tomchristiesetmessages: + msg340765
2019-04-24 08:48:27tomchristiecreate