classification
Title: Use Windows' certificate store for CA certs
Type: enhancement Stage: resolved
Components: Extension Modules Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, doko, dstufft, eric.araujo, exarkun, haypo, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2013-02-05 15:29 by christian.heimes, last changed 2013-12-04 07:24 by christian.heimes. This issue is now closed.

Files
File name Uploaded Description Edit
enumcertstore3.patch christian.heimes, 2013-06-08 17:51 review
enum_cert_trust2.patch christian.heimes, 2013-10-30 20:15 review
Messages (22)
msg181445 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-02-05 15:29
I found a recipe how to access the Windows certificate store and dump its content as PEM. The code doesn't look complicated and could be added to _ssl.c

http://fixunix.com/openssl/254866-re-can-openssl-use-windows-certificate-store.html
msg181459 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-02-05 16:43
Isn’t this part of #13655?  One feature is usually discussed for all platforms in one bug report.  (Sorry for all the bureaucracy in your recent reports, but it helps keep things manageable :)
msg181463 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-02-05 17:30
I like to split up tasks in small subtasks.

It's true that #13655 benefits from this feature but it can be implemented without this ticket. This enhancement also requires some addition to API and bindings to Windows' crypt32.dll. It might be inappropriate to add it to #13655 because we need to backport #13655 to Python 2.6 to 3.3.
msg181467 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-02-05 18:47
Sounds promising. Do you think this should be hooked into SSLContext.set_default_verify_paths, or be exposed as a separate method?
msg190743 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2013-06-07 11:47
> Sounds promising. Do you think this should be hooked into SSLContext.set_default_verify_paths, or be exposed as a separate method?

If there were an API which exposed the certificate material, then this would be more useful to libraries trying to do other things (present debugging information, use an alternate SSL implementation *wink*, etc).  If this is *only* wrapped up inside set_default_verify_paths then many of these extra things are impossible with a seconding binding to the same API.
msg190744 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-06-07 12:56
Yes, I'm planing to expose the low level API. I prefer to do as much work in Python space as possible. The information is just too useful to 3rd parties, too.

I'm thinking about one low level function that interfaces Windows's cert store. The rest can be build on top of this function and #18138.


enum_system_store(store_name, cert_type="certificate") -> [(cert_data, encoding_type), ...]

store_name:
  name of the store (e.g. "CA", "MY", "ROOT"), see http://msdn.microsoft.com/en-us/library/windows/desktop/aa376560%28v=vs.85%29.aspx
cert_type:
  "certificate" or "crl"
data:
  certificate bytes (as far as I know the certs are stored in DER format)
encoding_type:
  integer encoding X509_ASN_ENCODING or PKCS_7_ASN_ENCODING
msg190753 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-06-07 15:28
First patch. I have not yet verified that the return data can be loaded by openssl. Also I need to verify the error paths and add some tests, too.
msg190757 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-06-07 15:44
I fixed a ref leak and added some tests.
msg190812 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-06-08 17:51
New patch with fixed doc string and indention.

http://msdn.microsoft.com/en-us/library/windows/desktop/aa377189%28v=vs.85%29.aspx explains how encoding type shall be interpreted. I haven't seen PKCS#7 certs on my Windows system, though.

Instead of a flag I could also return a string: "CERTIFICATE" for X509_ASN_ENCODING cert, "X509 CRL" for X509_ASN_ENCODING CRL or "PKCS7" for PKCS#7 encoded certs.
msg190864 - (view) Author: Roundup Robot (python-dev) Date: 2013-06-09 17:03
New changeset 10d325f674f5 by Christian Heimes in branch 'default':
Issue #17134: Add ssl.enum_cert_store() as interface to Windows' cert store.
http://hg.python.org/cpython/rev/10d325f674f5
msg190865 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-06-09 17:14
> New changeset 10d325f674f5 by Christian Heimes in branch 'default':
> Issue #17134: Add ssl.enum_cert_store() as interface to Windows' cert store.
> http://hg.python.org/cpython/rev/10d325f674f5

I don't want to sound annoying, but I would have liked to review this 
before it goes in. Could it wait a few days? (I'm sure it can :-))
msg190894 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-06-10 08:57
Ezio already reviewed my code. But sure I can wait a couple of days. The second part of the patch depends on #18138 anyway.
msg193833 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-07-28 16:55
I guess I have to revise my patch and go throw Windows' crypto lookup functions... 

Automatic CA root certificate updates on Windows http://netsekure.org/2011/04/automatic-ca-root-certificate-updates-on-windows/
msg199341 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-10-09 20:29
The current implementation doesn't check the trust settings and purpose of certs.

CertGetCertificateContextProperty() with CERT_ENHKEY_USAGE_PROP_ID returns a ASN.1 structure. I just have to figure out how to parse the CTL_USAGE struct ...

http://msdn.microsoft.com/en-us/library/aa376079%28v=vs.85%29.aspx
http://msdn.microsoft.com/en-us/library/aa381493%28v=vs.85%29.aspx
http://www.alvestrand.no/objectid/1.3.6.1.5.5.7.3.html
msg200529 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-10-20 01:08
The new patch splits up the one function into enum_certificates() and enum_crls(). enum_certificates() now returns also trust settings for the certificate. Internally it maps the most common OIDs to human readable names.

The patch comes without doc updates yet.
msg201778 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-10-30 20:15
Here is a simplified version of my patch with doc updates.

Changes:

- Different functions for certs and CRLs: enum_certificates() / enum_crls()

- encoding is now a string ('x509_asn' or 'pkcs_7_asn')

- for certificates trust information is either a set of OIDs or True. The OIDs can be interpreter with the new functions #19448.

Both functions are intended to be low level interfaces to Window's cert store.
msg203153 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-11-17 13:57
The feature is not yet production-ready but part of the feature is already in 3.4. It depends on #19448 and #16487, too. What shall I do about it?
msg203709 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-22 00:51
New changeset 9adcb61ea741 by Christian Heimes in branch 'default':
Issue #17134: Finalize interface to Windows' certificate store. Cert and
http://hg.python.org/cpython/rev/9adcb61ea741
msg203736 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-22 10:59
The test is failing:

http://buildbot.python.org/all/builders/x86%20Windows%20Server%202003%20%5BSB%5D%203.x/builds/1758/steps/test/logs/stdio

======================================================================
FAIL: test_enum_certificates (test.test_ssl.BasicSocketTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "E:\Data\buildslave\cpython\3.x.snakebite-win2k3r2sp2-x86\build\lib\test\test_ssl.py", line 553, in test_enum_certificates
    self.assertIn(serverAuth, names)
AssertionError: '1.3.6.1.5.5.7.3.1' not found in {'1.3.6.1.5.5.7.3.3', '1.3.6.1.4.1.311.10.3.5', '2.16.840.1.113730.4.1', '2.16.840.1.113733.1.8.1'}

----------------------------------------------------------------------
msg203753 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-11-22 13:03
That's strange. It looks like the Win2k box has no root CA certs for serverAuth installed whatsoever. I'm adding Matthias to this ticket.
msg203781 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-22 15:14
New changeset de65df13ed50 by Christian Heimes in branch 'default':
Issue #17134: check certs of CA and ROOT system store
http://hg.python.org/cpython/rev/de65df13ed50
msg205202 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-12-04 07:24
The tests are passing again. Thanks!
History
Date User Action Args
2013-12-04 07:24:30christian.heimessetstatus: open -> closed
resolution: fixed
messages: + msg205202
2013-11-22 15:14:02python-devsetmessages: + msg203781
2013-11-22 13:03:30christian.heimessetnosy: + doko
messages: + msg203753
2013-11-22 10:59:45hayposetstatus: closed -> open

nosy: + haypo
messages: + msg203736

resolution: fixed -> (no value)
2013-11-22 01:35:12christian.heimessetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2013-11-22 00:51:40python-devsetmessages: + msg203709
2013-11-17 13:57:40christian.heimessetmessages: + msg203153
2013-10-30 20:15:01christian.heimessetfiles: + enum_cert_trust2.patch

messages: + msg201778
2013-10-30 20:11:34christian.heimessetfiles: - enumcertstore2.patch
2013-10-30 20:11:28christian.heimessetfiles: - enumcertstore.patch
2013-10-30 20:11:18christian.heimessetfiles: - enum_cert_trust.patch
2013-10-20 01:09:18christian.heimessetfiles: - certstore.cpp
2013-10-20 01:09:02christian.heimessetfiles: + enum_cert_trust.patch

messages: + msg200529
2013-10-09 20:29:06christian.heimessetmessages: + msg199341
2013-08-24 22:25:38dstufftsetnosy: + dstufft
2013-07-28 16:55:12christian.heimessetmessages: + msg193833
2013-06-10 08:57:33christian.heimessetmessages: + msg190894
2013-06-09 17:14:08pitrousetmessages: + msg190865
2013-06-09 17:03:41python-devsetnosy: + python-dev
messages: + msg190864
2013-06-08 17:51:44christian.heimessetfiles: + enumcertstore3.patch

messages: + msg190812
2013-06-07 15:44:40christian.heimessetfiles: + enumcertstore2.patch

messages: + msg190757
2013-06-07 15:28:19christian.heimessetfiles: + enumcertstore.patch
keywords: + patch
messages: + msg190753

stage: needs patch -> patch review
2013-06-07 12:56:22christian.heimessetmessages: + msg190744
2013-06-07 11:47:29exarkunsetnosy: + exarkun
messages: + msg190743
2013-02-05 18:47:12pitrousetmessages: + msg181467
2013-02-05 17:30:50christian.heimessetmessages: + msg181463
2013-02-05 16:43:02eric.araujosetnosy: + eric.araujo
messages: + msg181459
2013-02-05 16:23:16pitrousetnosy: + pitrou
2013-02-05 15:32:18christian.heimessetfiles: + certstore.cpp
2013-02-05 15:29:03christian.heimescreate