classification
Title: Memory leak in OpenSSL thread state
Type: resource usage Stage: resolved
Components: Extension Modules, SSL Versions: Python 3.7, Python 3.6, Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes
Priority: normal Keywords:

Created on 2016-11-02 12:01 by christian.heimes, last changed 2017-09-07 20:39 by christian.heimes. This issue is now closed.

Messages (2)
msg279922 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-11-02 12:01
Quote from https://github.com/curl/curl/issues/964

---
I wrote to Matt Caswell from openssl.org about this memleah, and he answered:
OpenSSL maintains a separate error queue for each thread. On each queue there can be
multiple errors. ERR_get_state() does not add any errors to the queue it
merely returns the ERR_STATE (i.e. the queue) for the current thread.
If the current thread has no queue then ERR_get_state() will create one.

ERR_clear_error() removes all the errors that are on the current
thread's queue. It does not deallocate the current thread's queue.

ERR_remove_thread_state() deallocates the specified thread's queue.

The mem leaks you are seeing are almost certainly because the queues for
your app's threads have not been deallocated.
---

The memory leak only affects OpenSSL 1.0.2 and older. OpenSSL 1.1.0 takes care of threading, locking and thread local resources itself.
msg301626 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-07 20:39
It's rather painful to fix the issue. Since the memory leak only affects users that create and destroy a lot of threads and the bug has been addressed by OpenSSL 1.1.0, I won't fix it.

If users are running into the issue, they should rather update to more recent OpenSSL versions. The 1.1.0 series uses proper thread local storage.
History
Date User Action Args
2017-09-07 20:39:37christian.heimessetstatus: open -> closed
versions: - Python 3.5
messages: + msg301626

resolution: wont fix
stage: test needed -> resolved
2016-11-02 12:01:29christian.heimescreate