classification
Title: SSLContext.load_verify_locations behavior on Windows and OSX
Type: behavior Stage: resolved
Components: Extension Modules, Library (Lib) Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alex, christian.heimes, dstufft, giampaolo.rodola, hynek, janssen, ncoghlan, ned.deily, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2014-09-20 10:35 by christian.heimes, last changed 2014-10-03 21:34 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
win32_load_SSL_CERT_env.patch christian.heimes, 2014-09-20 10:35 review
issue22449.diff alex, 2014-10-03 20:50 review
Messages (7)
msg227150 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-09-20 10:35
The behavior of SSLContext.load_verify_locations is rather inconsistent across platforms:

On most POSIX platforms (Linux, BSD, non-Apple builds of OpenSSL) it loads certificates from predefined locations. The locations are defined during compile time and usually differ between vendors and platforms. My WiP "Improve TLS/SSL support" PEP lists all common locations and the packages that offer the certs. On these platforms SSL_CERT_DIR and SSL_CERT_FILE overwrite the location.

On Windows SSL_CERT_DIR and SSL_CERT_FILE are never taken into account by SSLContext.load_verify_locations because it doesn't call SSLContext.set_default_verify_paths(). The attached patch is a semi-fix for the problem. With the patch certs from SSL_CERT_DIR and SSL_CERT_FILE are only *added* to trusted root CA certs. The certs from Windows' cert store 'CA' and 'ROOT' are still loaded.

On OSX with Apple's custom build of OpenSSL SSL_CERT_DIR and SSL_CERT_FILE take effect. But there is a twist! In case a root CA cert is not found Apple's Trust Evaluation Agent (TEA) kicks in and looks up certs from Apple's keychain. It's almost the same situation as on Windows but more magical. In order to disable TEA one has to set the env var OPENSSL_X509_TEA_DISABLE=1 *before* the first cert is validated. After that the env var has no effect as the value is cached. Hynek has documted it in his blog: https://hynek.me/articles/apple-openssl-verification-surprises/
msg227203 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2014-09-21 04:55
Does this effect anything besides causing SSL_CERT_DIR and SSL_CERT_FILE to be respected?
msg227209 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-09-21 10:41
'to respect' is the wrong term. </nitpicking>

The patch makes the code use the two env vars. Even with my patch the function doesn't respect SSL_CERT_DIR and SSL_CERT_FILE because Windows' cert store is still in use. On Linux and BSD the two env vars replace the default locations.
msg227211 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-21 11:15
Do we have tests for SSL_CERT_DIR and SSL_CERT_FILE being "respected"?
Otherwise, it might make sense to add one.
msg227215 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-09-21 12:39
There is only a test case for ssl.get_default_verify_paths(), but not for SSLContext or SSLSocket.
msg228375 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2014-10-03 20:50
Attached patch also adds a test -- I'm not convinced this is the best way to test this, but I don't see any other way either.
msg228389 - (view) Author: Roundup Robot (python-dev) Date: 2014-10-03 21:34
New changeset d9c52836aec8 by Benjamin Peterson in branch '2.7':
also use openssl envvars to find certs on windows (closes #22449)
https://hg.python.org/cpython/rev/d9c52836aec8

New changeset e1f453e13f8d by Benjamin Peterson in branch '3.4':
also use openssl envvars to find certs on windows (closes #22449)
https://hg.python.org/cpython/rev/e1f453e13f8d

New changeset f1113c568c60 by Benjamin Peterson in branch 'default':
merge 3.4 (#22449)
https://hg.python.org/cpython/rev/f1113c568c60
History
Date User Action Args
2014-10-03 21:34:23python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg228389

resolution: fixed
stage: needs patch -> resolved
2014-10-03 20:50:07alexsetfiles: + issue22449.diff

messages: + msg228375
2014-09-21 12:39:32christian.heimessetmessages: + msg227215
2014-09-21 11:15:13pitrousetmessages: + msg227211
2014-09-21 10:41:14christian.heimessetmessages: + msg227209
2014-09-21 04:55:50alexsetmessages: + msg227203
2014-09-20 21:51:39ned.deilysetnosy: + ned.deily
2014-09-20 10:35:39christian.heimescreate