classification
Title: SSLContext.load_verify_locations leaks memory on Linux in async code
Type: resource usage Stage:
Components: asyncio, Library (Lib), SSL Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Recursing, asvetlov, christian.heimes, yselivanov
Priority: normal Keywords:

Created on 2020-05-22 09:29 by Recursing, last changed 2020-05-22 10:32 by Recursing.

Messages (6)
msg369573 - (view) Author: (Recursing) * Date: 2020-05-22 09:29
Minimal code to reproduce:
```
import ssl
import certifi
import gc
import asyncio


ca_path = certifi.where()
async def make_async_context() -> None:
    context = ssl.SSLContext(ssl.PROTOCOL_TLS)
    context.load_verify_locations(ca_path)
    await asyncio.sleep(1)


async def main(n: int) -> None:
    await asyncio.wait([make_async_context() for _ in range(n)])


gc.collect()
asyncio.run(main(2000))
input("Finished run, still using lots of memory :(")
gc.collect()
input("gc.collect() does not help :(")
```

Running this code on several linux machines (with python from 3.6.9 to 3.9.0a5, and openSSL from 1.1.1  11 Sep 2018 to 1.1.1g  21 Apr 2020) causes a significant memory leak, while on windows memory usage peaks around 1 GB but gets freed
msg369578 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-05-22 09:40
Does it also leak without asyncio?
msg369582 - (view) Author: (Recursing) * Date: 2020-05-22 09:55
Removing the `await asyncio.sleep(1)` removes the leak, while changing it to `await asyncio.sleep(0)` seems to keep it
msg369583 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-05-22 10:09
Without asyncio memory consumption stays low and stable for me:

$ ./python -m venv venv
$ ./venv/bin/pip install psutil
$ ./venv/bin/python
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.1.1g FIPS  21 Apr 2020'
>>> import psutil, ssl, os
>>> p = psutil.Process(os.getpid())
>>> cafile = ssl.get_default_verify_paths().cafile
>>> p.memory_info()
pmem(rss=14811136, vms=237223936, shared=8138752, text=2125824, lib=0, data=6701056, dirty=0)
>>> for i in range(1000):
...     ssl.SSLContext(ssl.PROTOCOL_TLS).load_verify_locations(cafile)
... 
>>> p.memory_info()
pmem(rss=17489920, vms=238170112, shared=9863168, text=2125824, lib=0, data=7647232, dirty=0)
>>> for i in range(1000):
...     ssl.SSLContext(ssl.PROTOCOL_TLS).load_verify_locations(cafile)
... 
>>> p.memory_info()
pmem(rss=17489920, vms=238170112, shared=9863168, text=2125824, lib=0, data=7647232, dirty=0)
msg369584 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-05-22 10:17
When I run your example, RSS jumps from 20 MB to about 1,600 MB. There is almost no increase when I run the look several more times.

>>> p.memory_info()
pmem(rss=19902464, vms=240513024, shared=10014720, text=2125824, lib=0, data=9887744, dirty=0)
>>> asyncio.run(main(2000))
<stdin>:2: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11.
>>> p.memory_info()
pmem(rss=1608568832, vms=1829105664, shared=10014720, text=2125824, lib=0, data=1598480384, dirty=0)
>>> asyncio.run(main(2000))
>>> p.memory_info()
pmem(rss=1608835072, vms=1829367808, shared=10014720, text=2125824, lib=0, data=1598742528, dirty=0)
>>> asyncio.run(main(2000))
>>> p.memory_info()
pmem(rss=1608601600, vms=1829367808, shared=10014720, text=2125824, lib=0, data=1598742528, dirty=0)


Why are you creating so many SSLContext objects any way? It's very inefficient and really not necessary. I recommend that you create one context in your application and reuse for all connection. You only ever need additional contexts for different configuration (protocol, verification, trust anchors, ...).
msg369586 - (view) Author: (Recursing) * Date: 2020-05-22 10:32
> Without asyncio memory consumption stays low and stable for me

Same for me

> RSS jumps from 20 MB to about 1,600 MB.

That is the memory consumption I observe as well, the issue is that it doesn't get freed on Linux

> There is almost no increase when I run the look several more times.

Same for me, but of course only if I exit the "async" context between runs

> Why are you creating so many SSLContext objects any way? It's very inefficient and really not necessary.

The original issue was observed in a very long running process (months), that occasionally needed a context and it was convenient to just create one every time (actually it creates an AsyncClient context https://github.com/encode/httpx/issues/978) even if it is relatively inefficient, it didn't really matter, but memory usage unexpectedly slowly grew to 1 GB which was very unexpected
History
Date User Action Args
2020-05-22 10:32:24Recursingsetmessages: + msg369586
2020-05-22 10:17:45christian.heimessetassignee: christian.heimes ->
messages: + msg369584
2020-05-22 10:09:47christian.heimessetmessages: + msg369583
2020-05-22 09:55:53Recursingsetmessages: + msg369582
2020-05-22 09:40:48christian.heimessetmessages: + msg369578
2020-05-22 09:29:10Recursingcreate