This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: SSLContext.load_verify_locations behavior on Windows and OSX
Type: behavior Stage: resolved
Components: Extension Modules, Library (Lib) Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: LincolnPuzey, alex, christian.heimes, dstufft, giampaolo.rodola, hynek, janssen, lukasz.langa, miss-islington, ncoghlan, ned.deily, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2014-09-20 10:35 by christian.heimes, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
win32_load_SSL_CERT_env.patch christian.heimes, 2014-09-20 10:35 review
issue22449.diff alex, 2014-10-03 20:50 review
Pull Requests
URL Status Linked Edit
PR 28947 merged LincolnPuzey, 2021-10-14 12:56
PR 29373 merged miss-islington, 2021-11-02 21:27
PR 29374 merged miss-islington, 2021-11-02 21:27
Messages (10)
msg227150 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-09-20 10:35
The behavior of SSLContext.load_verify_locations is rather inconsistent across platforms:

On most POSIX platforms (Linux, BSD, non-Apple builds of OpenSSL) it loads certificates from predefined locations. The locations are defined during compile time and usually differ between vendors and platforms. My WiP "Improve TLS/SSL support" PEP lists all common locations and the packages that offer the certs. On these platforms SSL_CERT_DIR and SSL_CERT_FILE overwrite the location.

On Windows SSL_CERT_DIR and SSL_CERT_FILE are never taken into account by SSLContext.load_verify_locations because it doesn't call SSLContext.set_default_verify_paths(). The attached patch is a semi-fix for the problem. With the patch certs from SSL_CERT_DIR and SSL_CERT_FILE are only *added* to trusted root CA certs. The certs from Windows' cert store 'CA' and 'ROOT' are still loaded.

On OSX with Apple's custom build of OpenSSL SSL_CERT_DIR and SSL_CERT_FILE take effect. But there is a twist! In case a root CA cert is not found Apple's Trust Evaluation Agent (TEA) kicks in and looks up certs from Apple's keychain. It's almost the same situation as on Windows but more magical. In order to disable TEA one has to set the env var OPENSSL_X509_TEA_DISABLE=1 *before* the first cert is validated. After that the env var has no effect as the value is cached. Hynek has documted it in his blog: https://hynek.me/articles/apple-openssl-verification-surprises/
msg227203 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2014-09-21 04:55
Does this effect anything besides causing SSL_CERT_DIR and SSL_CERT_FILE to be respected?
msg227209 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-09-21 10:41
'to respect' is the wrong term. </nitpicking>

The patch makes the code use the two env vars. Even with my patch the function doesn't respect SSL_CERT_DIR and SSL_CERT_FILE because Windows' cert store is still in use. On Linux and BSD the two env vars replace the default locations.
msg227211 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-21 11:15
Do we have tests for SSL_CERT_DIR and SSL_CERT_FILE being "respected"?
Otherwise, it might make sense to add one.
msg227215 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-09-21 12:39
There is only a test case for ssl.get_default_verify_paths(), but not for SSLContext or SSLSocket.
msg228375 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2014-10-03 20:50
Attached patch also adds a test -- I'm not convinced this is the best way to test this, but I don't see any other way either.
msg228389 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-10-03 21:34
New changeset d9c52836aec8 by Benjamin Peterson in branch '2.7':
also use openssl envvars to find certs on windows (closes #22449)
https://hg.python.org/cpython/rev/d9c52836aec8

New changeset e1f453e13f8d by Benjamin Peterson in branch '3.4':
also use openssl envvars to find certs on windows (closes #22449)
https://hg.python.org/cpython/rev/e1f453e13f8d

New changeset f1113c568c60 by Benjamin Peterson in branch 'default':
merge 3.4 (#22449)
https://hg.python.org/cpython/rev/f1113c568c60
msg405553 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-11-02 21:27
New changeset 3551bf16ee5c25b6349209dd30e032f0f3b9ace3 by LincolnPuzey in branch 'main':
bpo-45457: Minor fix to documentation for SSLContext.load_default_certs. (GH-28947)
https://github.com/python/cpython/commit/3551bf16ee5c25b6349209dd30e032f0f3b9ace3
msg405555 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-11-02 21:48
New changeset 71f602b54c9c5346d22a542e186746b06cccfc8e by Miss Islington (bot) in branch '3.10':
bpo-45457: Minor fix to documentation for SSLContext.load_default_certs. (GH-28947) (GH-29373)
https://github.com/python/cpython/commit/71f602b54c9c5346d22a542e186746b06cccfc8e
msg405557 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-11-02 21:53
New changeset 4ab6e524abd2d7f21c2d7a6eaee5be1f93baf140 by Miss Islington (bot) in branch '3.9':
bpo-45457: Minor fix to documentation for SSLContext.load_default_certs. (GH-28947) (GH-29374)
https://github.com/python/cpython/commit/4ab6e524abd2d7f21c2d7a6eaee5be1f93baf140
History
Date User Action Args
2022-04-11 14:58:08adminsetgithub: 66639
2021-11-02 21:53:50lukasz.langasetmessages: + msg405557
2021-11-02 21:48:00lukasz.langasetmessages: + msg405555
2021-11-02 21:27:59lukasz.langasetnosy: + lukasz.langa
messages: + msg405553
2021-11-02 21:27:56miss-islingtonsetpull_requests: + pull_request27634
2021-11-02 21:27:52miss-islingtonsetnosy: + miss-islington

pull_requests: + pull_request27632
2021-10-14 12:56:37LincolnPuzeysetnosy: + LincolnPuzey

pull_requests: + pull_request27238
2014-10-03 21:34:23python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg228389

resolution: fixed
stage: needs patch -> resolved
2014-10-03 20:50:07alexsetfiles: + issue22449.diff

messages: + msg228375
2014-09-21 12:39:32christian.heimessetmessages: + msg227215
2014-09-21 11:15:13pitrousetmessages: + msg227211
2014-09-21 10:41:14christian.heimessetmessages: + msg227209
2014-09-21 04:55:50alexsetmessages: + msg227203
2014-09-20 21:51:39ned.deilysetnosy: + ned.deily
2014-09-20 10:35:39christian.heimescreate