classification
Title: SSLSocket.getpeercertchain()
Type: enhancement Stage: patch review
Components: SSL Versions: Python 3.8
process
Status: open Resolution:
Dependencies: 18147 Superseder:
Assigned To: christian.heimes Nosy List: Hiroaki.Kawai, asmodai, chaen, chet, chrisburr, christian.heimes, dsoprea, dstufft, jcea, joernheissler, maker, miki725, mmasztalerczuk, njs, pitrou, underrun
Priority: normal Keywords: patch

Created on 2013-06-16 20:39 by christian.heimes, last changed 2020-01-13 17:10 by chrisburr.

Files
File name Uploaded Description Edit
ssl_peerchertchain.patch christian.heimes, 2013-06-16 20:39 review
ssl_peerchertchain2.patch christian.heimes, 2013-06-17 18:07 review
ssl_peercertchain3.patch mmasztalerczuk, 2016-10-11 15:42 review
Pull Requests
URL Status Linked Edit
PR 17938 open chrisburr, 2020-01-10 15:56
Messages (24)
msg191287 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-06-16 20:39
The patch implements a method getpeercertchain() on a SSLSocket. It returns the peer's certificate chain from the leaf cert to the root cert if available. It wraps SSL_get_peer_cert_chain().

SSL_get_peer_cert_chain() doesn't have to pull any additional data from the peer. The information is already exchanged for cert validation.
msg191352 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-06-17 18:07
As expected it is much harder to get the full certification chain from OpenSSL than I initially expected. SSL_get_peer_cert_chain() doesn't return the root CA's certificate. The new patch introduces a validation mode and uses X509_verify_cert(*X509_STORE_CTX) + X509_STORE_CTX_get1_chain() to build a full chain.
msg193424 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-07-20 23:42
From Rietveld review:

---
http://bugs.python.org/review/18233/diff/8422/Modules/_ssl.c#newcode1203
Modules/_ssl.c:1203: chain = X509_STORE_CTX_get1_chain(store_ctx);
This isn't appropriate for this method. Specifically, you are asking for
the peer cert chain, which purposefully does not include root CA certs
that you trust. What you are giving here a complete validate chain from
a peer cert to a trusted root. This is a valuable piece of information,
but should be returned via another method (perhaps exposed in python as
get1chain in SSLContext). But this method should always return the
result of SSL_get_peer_cert_chain if a peer cert chain is available.
---

You are making a good point. I'm either going to split it up into two function or provide a way to look up a cert by issuer.
msg199434 - (view) Author: Dustin Oprea (dsoprea) * Date: 2013-10-11 01:28
I was about to submit a feature request to add exactly this. The [second] patch works like a charm. When are you going to land on a particular resolution so that it can get committed in?



Dustin
msg199443 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-11 06:30
The patch needs a test, a proper doc, and reviewing.
msg199446 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-11 09:06
Sorry for the incorrect answer. I just noticed there was a test in the patch!
Further looking at it, I notice the new function is returning a tuple. Wouldn't it be better to return a list here?
msg199447 - (view) Author: Dustin Oprea (dsoprea) * Date: 2013-10-11 09:09
My two-cents is to leave it a tuple (why not?).



Dustin
msg203187 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-11-17 14:54
I'd rather return a list or tuple of X509 objects but #18369 won't be ready for 3.4. Ideas?
msg203189 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-11-17 15:00
@Dustin

> My two-cents is to leave it a tuple (why not?).

Because tuples are more used for struct-like data. Here we are returning an unknown number of homogenous objects, which generally calls for a list.

@Christian

> I'd rather return a list or tuple of X509 objects but #18369 won't be ready for 3.4. Ideas?

Unless the feature is really important to have right now, I think it's ok to defer it until we have X509 objects.
msg203190 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-11-17 15:04
It's just nice to have for debugging and extended verification.
msg205734 - (view) Author: Derek Wilson (underrun) Date: 2013-12-09 19:35
I could really use this sooner than later... and sometimes having a full-featured (or even secure) interface is not what you want.

Consider zmap and masscan etc and ssl mapping (similar to what the EFF did a couple years back - https://www.eff.org/observatory - but with the full chain instead of just the cert). The desire here would be low level, low overhead, no validation on the fly: All you want is the cert chain.

There are plenty of research and security applications where a simple wrapper around OpenSSL that returns DER bytes would be desirable. Please reconsider this patch for inclusion in 3.4 ...
msg244312 - (view) Author: Jeroen Ruigrok van der Werven (asmodai) * (Python committer) Date: 2015-05-28 15:58
Given that cryptography.io is fast becoming the solution for dealing with X.509 certificates on Python, I would like to add my vote to add my vote for this feature. Right now, getting the full chain in DER is what I am missing to complete a task at work.
msg244313 - (view) Author: Dustin Oprea (dsoprea) * Date: 2015-05-28 16:03
Forget it. This project is dead.

Dustin
On May 28, 2015 11:58 AM, "Jeroen Ruigrok van der Werven" <
report@bugs.python.org> wrote:

>
> Jeroen Ruigrok van der Werven added the comment:
>
> Given that cryptography.io is fast becoming the solution for dealing with
> X.509 certificates on Python, I would like to add my vote to add my vote
> for this feature. Right now, getting the full chain in DER is what I am
> missing to complete a task at work.
>
> ----------
> nosy: +asmodai
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue18233>
> _______________________________________
>
msg244314 - (view) Author: Dustin Oprea (dsoprea) * Date: 2015-05-28 16:07
Disregard. I thought this was something else.
msg278494 - (view) Author: Mariusz Masztalerczuk (mmasztalerczuk) * Date: 2016-10-11 15:42
Hello :)

I'm not sure why patches created by christian.heimes is not merged to python, but because last patch was created in 2013, I've created a new version of this patch.

What do you think about it?
msg280348 - (view) Author: Mariusz Masztalerczuk (mmasztalerczuk) * Date: 2016-11-08 21:05
ping! :) 
Could someone look at my changes? :)
msg293570 - (view) Author: Chet Nichols III (chet) Date: 2017-05-12 20:48
Is this dead at this point? Just stumbled upon it, and I'm hopeful that maybe there's still a chance, since it's still `open`. :)
msg293577 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-05-12 21:45
The ticket is dead for a very good reason. Past me was not clever enough and didn't know about the difference between the cert chain sent by the peer and the actual trust chain. The peer's cert chain is not trustworthy and must *only* be used to build the actual trust chain. X.509 chain trust chain construction is a tricky business.

Although I thought that peer cert chain is a useful piece of information, it is also dangerous. It's simply not trustworthy. In virtually all cases you want to know the chain of certificates that leads from a local trust anchor to the end-entity cert. In most cases it just happens to be the same (excluding root CA). But that's not reliable.
msg293580 - (view) Author: Dustin Oprea (dsoprea) * Date: 2017-05-12 22:07
Thanks for expounding on this, Christian. Assuming your assertions are
correct, this makes perfect sense.

Can anyone listening close this?

On May 12, 2017 17:45, "Christian Heimes" <report@bugs.python.org> wrote:

Christian Heimes added the comment:

The ticket is dead for a very good reason. Past me was not clever enough
and didn't know about the difference between the cert chain sent by the
peer and the actual trust chain. The peer's cert chain is not trustworthy
and must *only* be used to build the actual trust chain. X.509 chain trust
chain construction is a tricky business.

Although I thought that peer cert chain is a useful piece of information,
it is also dangerous. It's simply not trustworthy. In virtually all cases
you want to know the chain of certificates that leads from a local trust
anchor to the end-entity cert. In most cases it just happens to be the same
(excluding root CA). But that's not reliable.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue18233>
_______________________________________
msg293590 - (view) Author: Chet Nichols III (chet) Date: 2017-05-12 22:52
Oh yeah, definitely not trustworthy at all. In my case, I am not processing the peer chain to actually verify trust, but I am still interested in inspecting the chain.

Dangerous or not, and regardless of what almost all people should *actually* be doing, SSL_get_peer_cert_chain exists for a reason, just like SSL_get_peer_certificate exists for a reason. If Python includes a standard SSL library, it should be transparent in the interface it offers, for the mere reason that the library becomes more powerful.

If the overall consensus is that the library should protect most people against common pitfalls and security mistakes, then I guess that's the route to continue on. However, I would be disappointed that we would be blacklisting  the exposure of underlying library features based on the mere belief that people don't understand them enough!
msg293778 - (view) Author: Jörn Heissler (joernheissler) * Date: 2017-05-16 18:53
Hi,
I'd like to see this feature too.

My use case is a monitoring script to check the life time of the server certificate, including the chain. I would prefer to have a wrapper around SSL_get_peer_cert_chain.
I understand that this is *not* a verified chain. That's okay.

openssl-1.1 added a new function SSL_get0_verified_chain which may be safer for most applications. Is there any real difference to X509_STORE_CTX_get1_chain?

If you're worried about people misusing these functions, add a warning in the docs and point them to "get_peer_verified_chain"?
msg301525 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-06 21:42
Yes, from an application perspective there is an import difference between X509_STORE_CTX_get1_chain() and SSL_get0_verified_chain(). X509_STORE_CTX is a temporary object. It is only available during the handshake and while the trust chain is built and verified. Once the chain is verified, it is no longer available.

SSL_get0_verified_chain() sounds like an actual good solution. Thanks for pointing it out.
msg324917 - (view) Author: chaen (chaen) Date: 2018-09-10 10:13
There is another very valid use case which is even described by an RFC: https://www.ietf.org/rfc/rfc3820.txt

And openssl supports this RFC. 

These proxy certificates are heavily used in the world of high energy physics computing, and having the get_peer_cert_chain exposed in python would allow to use standard tools. 

And any hope this would also be backported to the python 2.7 branch ?
msg357540 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2019-11-27 00:13
There's another important use case for this, that hasn't been discussed here. If you want to use openssl for TLS + the system trust store to verify certificates, then you need to disable openssl's certificate validation, perform the handshake, and then extract the certificate chain that there peer sent and pass it to the system native APIs to validate.

For this case, we don't need to do any validation or resolution on the chain – we just want to pull out the DER that the peer sent. AFAICT, the lack of this functionality is the one major blocker to using the system trust store with the 'ssl' module.
History
Date User Action Args
2020-01-13 17:10:31chrisburrsetnosy: + chrisburr
2020-01-10 15:56:01chrisburrsetpull_requests: + pull_request17346
2019-11-27 00:13:37njssetnosy: + njs
messages: + msg357540
2019-03-31 10:08:16Hiroaki.Kawaisetnosy: + Hiroaki.Kawai
2018-09-10 10:13:09chaensetnosy: + chaen
messages: + msg324917
2018-02-26 08:31:55christian.heimessetversions: + Python 3.8, - Python 3.7
2017-09-06 21:42:30christian.heimessetmessages: + msg301525
2017-05-16 18:53:15joernheisslersetnosy: + joernheissler
messages: + msg293778
2017-05-12 22:52:05chetsetmessages: + msg293590
2017-05-12 22:07:08dsopreasetmessages: + msg293580
2017-05-12 21:45:01christian.heimessetmessages: + msg293577
2017-05-12 20:48:23chetsetnosy: + chet
messages: + msg293570
2016-11-08 21:05:57mmasztalerczuksetmessages: + msg280348
2016-10-11 15:42:59mmasztalerczuksetfiles: + ssl_peercertchain3.patch
nosy: + mmasztalerczuk
messages: + msg278494

2016-09-15 07:48:34christian.heimessetassignee: christian.heimes
components: + SSL
2016-09-15 07:25:56christian.heimessetversions: - Python 3.6
2016-09-11 02:07:54miki725setnosy: + miki725
2016-09-08 15:42:23christian.heimessetversions: + Python 3.6, Python 3.7, - Python 3.4, Python 3.5
2016-06-12 11:23:15christian.heimessetassignee: christian.heimes -> (no value)
2015-05-28 16:07:07dsopreasetmessages: + msg244314
2015-05-28 16:03:38dsopreasetmessages: + msg244313
2015-05-28 15:58:45asmodaisetnosy: + asmodai
messages: + msg244312
2013-12-09 19:35:15underrunsetmessages: + msg205734
versions: + Python 3.4
2013-11-20 16:47:33christian.heimessetassignee: christian.heimes
2013-11-17 15:04:51christian.heimessetmessages: + msg203190
versions: + Python 3.5, - Python 3.4
2013-11-17 15:00:04pitrousetmessages: + msg203189
2013-11-17 14:54:20christian.heimessetmessages: + msg203187
2013-10-11 09:09:34dsopreasetmessages: + msg199447
2013-10-11 09:06:33pitrousetmessages: + msg199446
2013-10-11 06:30:31pitrousetnosy: + pitrou
messages: + msg199443
2013-10-11 01:28:42dsopreasetnosy: + dsoprea
messages: + msg199434
2013-08-24 22:37:35dstufftsetnosy: + dstufft
2013-07-24 15:26:34christian.heimeslinkissue18546 superseder
2013-07-20 23:42:20christian.heimessetmessages: + msg193424
2013-07-20 23:15:38underrunsetnosy: + underrun
2013-07-05 20:35:35makersetnosy: + maker
2013-06-20 02:26:38jceasetnosy: + jcea
2013-06-17 18:07:25christian.heimessetfiles: + ssl_peerchertchain2.patch

messages: + msg191352
2013-06-17 11:55:30pitrousetdependencies: + SSL: diagnostic functions to list loaded CA certs
2013-06-16 20:39:55christian.heimescreate