This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ssl.get_server_certificate should use SNI
Type: behavior Stage: resolved
Components: SSL Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, enki, miss-islington
Priority: normal Keywords: patch

Created on 2019-02-22 10:50 by enki, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 16820 merged juhovh, 2019-10-16 09:06
Messages (3)
msg336292 - (view) Author: Maciej Grela (enki) Date: 2019-02-22 10:50
The ssl.get_server_certificate function doesn't send SNI information causing an wrong certificate to be sent back by the server (or connection close in some cases). This can be seen when trying to use get_server_certificate against a site behind cloudflare. An example is provided below:

$ python3 -V
Python 3.7.2
$ python3 -c "import ssl; print(ssl.get_server_certificate(('www.mx.com',443)))" | openssl x509 -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            89:2a:bc:df:8a:f3:d6:f6:ae:c5:18:5a:78:ec:39:6e
    Signature Algorithm: ecdsa-with-SHA256
        Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO ECC Domain Validation Secure Server CA 2
        Validity
            Not Before: Dec 19 00:00:00 2018 GMT
            Not After : Jun 27 23:59:59 2019 GMT
        Subject: OU=Domain Control Validated, OU=PositiveSSL Multi-Domain, CN=ssl803013.cloudflaressl.com
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub: 
                    04:ff:c1:c3:f1:c0:8a:08:84:ad:e4:25:f6:c3:03:
                    1f:26:0a:b4:85:e0:65:0e:f5:8b:13:1e:21:b2:54:
                    94:8c:f3:ce:98:eb:cf:ff:ff:1d:3a:03:22:b1:7c:
                    5f:13:e5:09:1f:77:b0:e8:ac:bf:e6:6c:ea:cb:57:
                    df:e1:c8:14:da
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Authority Key Identifier: 
                keyid:40:09:61:67:F0:BC:83:71:4F:DE:12:08:2C:6F:D4:D4:2B:76:3D:96

            X509v3 Subject Key Identifier: 
                4B:F4:77:CD:FB:04:DC:0D:B2:A5:99:B8:6F:17:CC:80:DF:AE:59:DF
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Certificate Policies: 
                Policy: 1.3.6.1.4.1.6449.1.2.2.7
                  CPS: https://secure.comodo.com/CPS
                Policy: 2.23.140.1.2.1

            X509v3 CRL Distribution Points: 

                Full Name:
                  URI:http://crl.comodoca4.com/COMODOECCDomainValidationSecureServerCA2.crl

            Authority Information Access: 
                CA Issuers - URI:http://crt.comodoca4.com/COMODOECCDomainValidationSecureServerCA2.crt
                OCSP - URI:http://ocsp.comodoca4.com

            X509v3 Subject Alternative Name: 
                DNS:ssl803013.cloudflaressl.com, DNS:*.hscoscdn00.net, DNS:hscoscdn00.net
            1.3.6.1.4.1.11129.2.4.2: 
                ......u.......q...#...{G8W.
.R....d6.......g..P......F0D. ...0....J|..2I..}%.Q.P...Z....g.. ......e....j...Y^.Ti^..........].w.t~..1.3..!..%OBp...^B ..75y..{.V...g..Pv.....H0F.!..1#I..\.....#2...$...X....
......!...].{o..ud..6OV
Q.x...J_(....[!.
    Signature Algorithm: ecdsa-with-SHA256
         30:45:02:20:0c:8c:b6:ea:68:e4:d6:d6:18:95:50:8f:77:41:
         63:51:81:59:3b:1b:e6:38:47:88:f3:47:d5:b0:0b:03:c5:ba:
         02:21:00:d2:19:3f:71:e2:64:36:79:d1:4c:c9:98:fd:74:d7:
         32:53:f6:b4:de:09:65:d8:a0:60:85:eb:f1:1f:75:35:75
-----BEGIN CERTIFICATE-----
MIIFBzCCBK2gAwIBAgIRAIkqvN+K89b2rsUYWnjsOW4wCgYIKoZIzj0EAwIwgZIx
CzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNV
BAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMTgwNgYDVQQD
Ey9DT01PRE8gRUNDIERvbWFpbiBWYWxpZGF0aW9uIFNlY3VyZSBTZXJ2ZXIgQ0Eg
MjAeFw0xODEyMTkwMDAwMDBaFw0xOTA2MjcyMzU5NTlaMGwxITAfBgNVBAsTGERv
bWFpbiBDb250cm9sIFZhbGlkYXRlZDEhMB8GA1UECxMYUG9zaXRpdmVTU0wgTXVs
dGktRG9tYWluMSQwIgYDVQQDExtzc2w4MDMwMTMuY2xvdWRmbGFyZXNzbC5jb20w
WTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAT/wcPxwIoIhK3kJfbDAx8mCrSF4GUO
9YsTHiGyVJSM886Y68///x06AyKxfF8T5Qkfd7DorL/mbOrLV9/hyBTao4IDBzCC
AwMwHwYDVR0jBBgwFoAUQAlhZ/C8g3FP3hIILG/U1Ct2PZYwHQYDVR0OBBYEFEv0
d837BNwNsqWZuG8XzIDfrlnfMA4GA1UdDwEB/wQEAwIHgDAMBgNVHRMBAf8EAjAA
MB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjBPBgNVHSAESDBGMDoGCysG
AQQBsjEBAgIHMCswKQYIKwYBBQUHAgEWHWh0dHBzOi8vc2VjdXJlLmNvbW9kby5j
b20vQ1BTMAgGBmeBDAECATBWBgNVHR8ETzBNMEugSaBHhkVodHRwOi8vY3JsLmNv
bW9kb2NhNC5jb20vQ09NT0RPRUNDRG9tYWluVmFsaWRhdGlvblNlY3VyZVNlcnZl
ckNBMi5jcmwwgYgGCCsGAQUFBwEBBHwwejBRBggrBgEFBQcwAoZFaHR0cDovL2Ny
dC5jb21vZG9jYTQuY29tL0NPTU9ET0VDQ0RvbWFpblZhbGlkYXRpb25TZWN1cmVT
ZXJ2ZXJDQTIuY3J0MCUGCCsGAQUFBzABhhlodHRwOi8vb2NzcC5jb21vZG9jYTQu
Y29tMEgGA1UdEQRBMD+CG3NzbDgwMzAxMy5jbG91ZGZsYXJlc3NsLmNvbYIQKi5o
c2Nvc2NkbjAwLm5ldIIOaHNjb3NjZG4wMC5uZXQwggEEBgorBgEEAdZ5AgQCBIH1
BIHyAPAAdQC72d+8H4pxtZOUI5eqkntHOFeVCqtS6BqQlmQ2jh7RhQAAAWfEHVAd
AAAEAwBGMEQCIB73izCk6hPeSnyYAjJJD959JRpRHFD2EwNa3Bzo3mcUAiASBZfK
xLxlg5H302r4gOVZXrFUaV6Ylpy57rzdrvb4XQB3AHR+2oMxrTMQkSGcziVPQnDC
v/1eQiAIxjc1eeYQe8xWAAABZ8QdUHYAAAQDAEgwRgIhAMMxI0kXulz+sMgA7iMy
LrjbJLcZ6ViL4e71CpHc99etAiEAzKFdBXtvvJR1ZAj4Nk9WClGbeBmO7EpfKP+4
nMFbIQ4wCgYIKoZIzj0EAwIDSAAwRQIgDIy26mjk1tYYlVCPd0FjUYFZOxvmOEeI
80fVsAsDxboCIQDSGT9x4mQ2edFMyZj9dNcyU/a03gll2KBghevxH3U1dQ==
-----END CERTIFICATE-----


As you can see the certificate returned has CN=ssl803013.cloudflaressl.com, the proper certificate has CN=www.mx.com as you can compare with openssl output when SNI is sent:

$ openssl s_client -connect www.mx.com:443 -servername www.mx.com
CONNECTED(00000006)
depth=2 C = IE, O = Baltimore, OU = CyberTrust, CN = Baltimore CyberTrust Root
verify return:1
depth=1 C = US, ST = CA, L = San Francisco, O = "CloudFlare, Inc.", CN = CloudFlare Inc ECC CA-2
verify return:1
depth=0 C = US, ST = CA, L = San Francisco, O = "CloudFlare, Inc.", CN = www.mx.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=CA/L=San Francisco/O=CloudFlare, Inc./CN=www.mx.com
   i:/C=US/ST=CA/L=San Francisco/O=CloudFlare, Inc./CN=CloudFlare Inc ECC CA-2
 1 s:/C=US/ST=CA/L=San Francisco/O=CloudFlare, Inc./CN=CloudFlare Inc ECC CA-2
   i:/C=IE/O=Baltimore/OU=CyberTrust/CN=Baltimore CyberTrust Root
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEsTCCBFigAwIBAgIQAjcQsekTBs8hJemLjbikNDAKBggqhkjOPQQDAjBvMQsw
CQYDVQQGEwJVUzELMAkGA1UECBMCQ0ExFjAUBgNVBAcTDVNhbiBGcmFuY2lzY28x
GTAXBgNVBAoTEENsb3VkRmxhcmUsIEluYy4xIDAeBgNVBAMTF0Nsb3VkRmxhcmUg
SW5jIEVDQyBDQS0yMB4XDTE4MTAxODAwMDAwMFoXDTE5MTAxODEyMDAwMFowYjEL
MAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMRYwFAYDVQQHEw1TYW4gRnJhbmNpc2Nv
MRkwFwYDVQQKExBDbG91ZEZsYXJlLCBJbmMuMRMwEQYDVQQDEwp3d3cubXguY29t
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEpIwExa1SMy+gfeVr5vkDU6HP75tT
R/EXeti75Mp7M8+aWGjndy0frFF99sUbLd7eLU4AvcQ/55Q+IPI70czNmqOCAuEw
ggLdMB8GA1UdIwQYMBaAFD50LR/PRXUEfj/Aooc+TEODURPGMB0GA1UdDgQWBBTf
wVI3nkYN/6HbYZq9m7DuZqxxxzAVBgNVHREEDjAMggp3d3cubXguY29tMA4GA1Ud
DwEB/wQEAwIHgDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIweQYDVR0f
BHIwcDA2oDSgMoYwaHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0Nsb3VkRmxhcmVJ
bmNFQ0NDQTIuY3JsMDagNKAyhjBodHRwOi8vY3JsNC5kaWdpY2VydC5jb20vQ2xv
dWRGbGFyZUluY0VDQ0NBMi5jcmwwTAYDVR0gBEUwQzA3BglghkgBhv1sAQEwKjAo
BggrBgEFBQcCARYcaHR0cHM6Ly93d3cuZGlnaWNlcnQuY29tL0NQUzAIBgZngQwB
AgIwdgYIKwYBBQUHAQEEajBoMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdp
Y2VydC5jb20wQAYIKwYBBQUHMAKGNGh0dHA6Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNv
bS9DbG91ZEZsYXJlSW5jRUNDQ0EtMi5jcnQwDAYDVR0TAQH/BAIwADCCAQQGCisG
AQQB1nkCBAIEgfUEgfIA8AB2ALvZ37wfinG1k5Qjl6qSe0c4V5UKq1LoGpCWZDaO
HtGFAAABZoguW3YAAAQDAEcwRQIhANcAvb1Emol7u2k7LdfkMdUTL8DNU+HNWLZc
PjNrfYBfAiBqo3ixk0WfrJ/4X1Esr/DzpasP70RqlvNnhSQpKhUTEwB2AHR+2oMx
rTMQkSGcziVPQnDCv/1eQiAIxjc1eeYQe8xWAAABZoguW2oAAAQDAEcwRQIhAPKv
tl3iaYpmRBBN9rmcafB3FGBAdQ8ta5y8xPWjpn1cAiAJW45I6ekD3Afp0Nri+qOO
426qMXl9lJTkI+h7seRfEjAKBggqhkjOPQQDAgNHADBEAiBJVUN9XUYl0Hy/f7yn
K+ximLR+8xINlban5hHj0PeghAIgXIoKNIKAl7r3la1J1KnWAfaoGgOo86hgSGLv
b2tH1ps=
-----END CERTIFICATE-----
subject=/C=US/ST=CA/L=San Francisco/O=CloudFlare, Inc./CN=www.mx.com
issuer=/C=US/ST=CA/L=San Francisco/O=CloudFlare, Inc./CN=CloudFlare Inc ECC CA-2
---
No client certificate CA names sent
Server Temp Key: ECDH, X25519, 253 bits
---
SSL handshake has read 2585 bytes and written 304 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-ECDSA-CHACHA20-POLY1305
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-CHACHA20-POLY1305
    Session-ID: 2A4A8733FAEB281F19328C80B975907C6DB0FBE37B1D4A0A31D45C21504F10B4
    Session-ID-ctx: 
    Master-Key: F2F3881060640EF36D03DBAD46EBA8C3626D0CB2BE1253376DB98668E037D8C3ED03CE468DA5C5C86FFACB1C0C0943C6
    TLS session ticket lifetime hint: 64800 (seconds)
    TLS session ticket:


The problem lies within Lib/ssl.py get_server_certificate function:

def get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None):
    """Retrieve the certificate from the server at the specified address,
    and return it as a PEM-encoded string.
    If 'ca_certs' is specified, validate the server cert against it.
    If 'ssl_version' is specified, use it in the connection attempt."""

    host, port = addr
    if ca_certs is not None:
        cert_reqs = CERT_REQUIRED
    else:
        cert_reqs = CERT_NONE
    context = _create_stdlib_context(ssl_version,
                                     cert_reqs=cert_reqs,
                                     cafile=ca_certs)
    with  create_connection(addr) as sock:
        with context.wrap_socket(sock) as sslsock:
            dercert = sslsock.getpeercert(True)
    return DER_cert_to_PEM_cert(dercert)

The wrap_socket function should be called with server_hostname parameter to send the SNI information.
msg391331 - (view) Author: miss-islington (miss-islington) Date: 2021-04-18 11:11
New changeset 49fdf118aeda891401d638ac32296c7d55d54678 by juhovh in branch 'master':
bpo-36076: Add SNI support to ssl.get_server_certificate. (GH-16820)
https://github.com/python/cpython/commit/49fdf118aeda891401d638ac32296c7d55d54678
msg391332 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-18 11:29
Thanks!
History
Date User Action Args
2022-04-11 14:59:11adminsetgithub: 80257
2021-04-18 11:29:25christian.heimessetstatus: open -> closed
versions: + Python 3.10, - Python 3.7
messages: + msg391332

resolution: fixed
stage: patch review -> resolved
2021-04-18 11:11:51miss-islingtonsetnosy: + miss-islington
messages: + msg391331
2019-10-16 09:06:32juhovhsetkeywords: + patch
stage: patch review
pull_requests: + pull_request16372
2019-02-22 10:50:37enkicreate