classification
Title: test_ssl.test_get_server_certificate() should use PROTOCOL_SSLv23, not PROTOCOL_SSLv3
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.3, Python 3.1, Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: GreenKey, christian.heimes, ddvento@ucar.edu, haypo, koobs, pitrou, python-dev
Priority: normal Keywords:

Created on 2014-03-12 11:20 by haypo, last changed 2015-01-06 11:24 by python-dev. This issue is now closed.

Messages (13)
msg213248 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-03-12 11:20
======================================================================
ERROR: test_get_server_certificate (test.test_ssl.NetworkedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/haypo/prog/python/default/Lib/test/test_ssl.py", line 1373, in test_get_server_certificate
    _test_get_server_certificate('svn.python.org', 443, SVN_PYTHON_ORG_ROOT_CERT)
  File "/home/haypo/prog/python/default/Lib/test/test_ssl.py", line 1354, in _test_get_server_certificate
    pem = ssl.get_server_certificate((host, port))
  File "/home/haypo/prog/python/default/Lib/ssl.py", line 902, in get_server_certificate
    with context.wrap_socket(sock) as sslsock:
  File "/home/haypo/prog/python/default/Lib/ssl.py", line 344, in wrap_socket
    _context=self)
  File "/home/haypo/prog/python/default/Lib/ssl.py", line 540, in __init__
    self.do_handshake()
  File "/home/haypo/prog/python/default/Lib/ssl.py", line 767, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:598)

----------------------------------------------------------------------

Extract of the current CA cert of svn.python.org:
---
$ openssl x509 -in Lib/test/https_svn_python_org_root.pem -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 0 (0x0)
    Signature Algorithm: md5WithRSAEncryption
        Issuer: O=Root CA, OU=http://www.cacert.org, CN=CA Cert Signing Authority/emailAddress=support@cacert.org
        Validity
            Not Before: Mar 30 12:29:49 2003 GMT
            Not After : Mar 29 12:29:49 2033 GMT
        Subject: O=Root CA, OU=http://www.cacert.org, CN=CA Cert Signing Authority/emailAddress=support@cacert.org
...
---

Lib/test/https_svn_python_org_root.pem is identical to http://www.cacert.org/certs/root.crt: root certificate of cacert.org authority.
msg213249 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-03-12 11:24
Debug with OpenSSL command line:

$ openssl s_client -connect svn.python.org:443 -CAfile Lib/test/https_svn_python_org_root.pem 
CONNECTED(00000003)
depth=1 O = Root CA, OU = http://www.cacert.org, CN = CA Cert Signing Authority, emailAddress = support@cacert.org
verify return:1
depth=0 CN = svn.python.org
verify return:1
---
Certificate chain
 0 s:/CN=svn.python.org
   i:/O=Root CA/OU=http://www.cacert.org/CN=CA Cert Signing Authority/emailAddress=support@cacert.org
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEzzCCAregAwIBAgIDDkGJMA0GCSqGSIb3DQEBBQUAMHkxEDAOBgNVBAoTB1Jv
b3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZ
Q0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJARYSc3VwcG9y
dEBjYWNlcnQub3JnMB4XDTEzMTIyNDIwMjgzMloXDTE1MTIyNDIwMjgzMlowGTEX
MBUGA1UEAxMOc3ZuLnB5dGhvbi5vcmcwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQDM6un3wTW9+HVJ7KC+/GwL0KAxehug0tw2YoSSX+TGxLyr9AUtBHQk
hCNWhRLewa0WMOY6hxIIQY1Hp6vreDiCbBehjVkEAydlKBzaAsgYCEbCC/ZaMzhv
aaFAiLVeaxAKJsBGUJNz5hGgzd67A6SGz+XK7qDWig4NR5eFrsr3DvjyEM7txMiG
gftGWLkadOuqUQsI20AykBGi+RxmrQIwqO2svGmje89DsWVILdP37PssM2zqRonh
4fUKooei3L43tXbTdHayXc9NtFS7q8T4eUlyWaD+BtP80QQOQFFvi+qZpme9bmYI
7YPX+e86lZtxAKM9nWrP93qc+2nS0MsHAgMBAAGjgb8wgbwwDAYDVR0TAQH/BAIw
ADAOBgNVHQ8BAf8EBAMCA6gwNAYDVR0lBC0wKwYIKwYBBQUHAwIGCCsGAQUFBwMB
BglghkgBhvhCBAEGCisGAQQBgjcKAwMwMwYIKwYBBQUHAQEEJzAlMCMGCCsGAQUF
BzABhhdodHRwOi8vb2NzcC5jYWNlcnQub3JnLzAxBgNVHR8EKjAoMCagJKAihiBo
dHRwOi8vY3JsLmNhY2VydC5vcmcvcmV2b2tlLmNybDANBgkqhkiG9w0BAQUFAAOC
AgEAvq2TlM5voqqwlfbEIwdNM3RlqHattH/h2Pqkr5FV6nynhcNyP9wBJyS5mdlt
+tj1Fy6oV+iN0s8VglrwhsmYN2pbJkCJZcNgrhFTiOvZ3HJ0jxkQ8TaROfcH1RnZ
q45DcbSnryiIDBQNYxmUf/bE6Ce+48fiOJpTRtrC8iWfz4J/JHYZ5FVKx+SsC67o
E1iCT2/r1PSvzTg/bohL1kX6Aj3H3UqlXDuDybyHRrTn9kUwCgXK2h4x/qae6xjo
Gy++gtzg4XlJEAh65znJ2RlAYi3lvhCls+viR5vSIXBEXmBPFNonGnPtGRyx6Tii
5ncSUP+bp6aaRZb0qEi0k1R5XYPt5Hm7T/h4IOAKlx7gPMMLOXqbrlWpjgwhfysI
f/KkgGBq7nTsC6WeaW/QUQtdXqWULrK+nINY+s/CRX5UxdAWpqMxqgNkP3zef9yO
etgfpR+6NArjiLNcfJO2yGZTcXn1H2gewTcxwmc+QVWday9HcS3paMpot8BwWUpI
LzOLGTeNH+rQCUAKJHAiK3Ogee+hka4icIN7cKOIIVfZ/XBj1Ex7zjc3j72axNba
S1buqvrm12YCE5+xAjYwU/Nrl4HNFKCoPW7qfEaEuwp49pieAIOnXa82rCh/UdAv
dSj5JpEvxjTuZdVfbk2VUc/z2OeLzFlrQRJsOt1MisY0Aoc=
-----END CERTIFICATE-----
subject=/CN=svn.python.org
issuer=/O=Root CA/OU=http://www.cacert.org/CN=CA Cert Signing Authority/emailAddress=support@cacert.org
---
No client certificate CA names sent
---
SSL handshake has read 2112 bytes and written 439 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : DHE-RSA-AES256-GCM-SHA384
    Session-ID: A8A1C5EC36ACD8FF0120271C1F16BDE7720FD0DC69871D1BE394A22309C09FE5
    Session-ID-ctx: 
    Master-Key: EF899D1961B522E380366F8832E7DF72AB56B9D76388B80A907637E2948D94514CADE5885CA3AF11B40F43E14F42ED92
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 7f 00 da de 6a a5 79 fa-9e 83 e8 20 1c 75 ef 54   ....j.y.... .u.T
    0010 - 34 43 3a 0a 50 0c f7 00-31 79 02 38 9f 8e 49 d6   4C:.P...1y.8..I.
    0020 - f1 25 57 c7 4f 97 f3 3a-a3 fa 8b 1b 8a 3b 5f e9   .%W.O..:.....;_.
    0030 - 6b ba 89 e1 db ba a4 e2-1d 3c f6 e4 cf d6 2f 54   k........<..../T
    0040 - 82 4f 24 1c 93 44 c3 ff-79 3d 67 34 27 48 34 db   .O$..D..y=g4'H4.
    0050 - 5b b4 a2 30 5b 16 e7 b5-ba ee 89 0a c5 89 a9 9a   [..0[...........
    0060 - fe 32 77 23 b3 b5 b4 fb-63 b5 87 d4 20 b2 18 7f   .2w#....c... ...
    0070 - 45 4e e6 f5 6f bd f4 24-80 b3 37 fd b5 83 2e 87   EN..o..$..7.....
    0080 - a1 b2 bb 4f b0 e3 7a 28-26 4a 71 3a 92 5e d1 aa   ...O..z(&Jq:.^..
    0090 - be 77 67 79 ad ea d0 c8-d4 d2 8a 44 f0 f5 ec c5   .wgy.......D....
    00a0 - 00 0b 5b 82 c1 51 45 ef-d0 6c fb 03 46 3f b1 e2   ..[..QE..l..F?..
    00b0 - 54 f8 27 4b 8f a0 e0 2e-7b 4f d8 42 29 76 74 b3   T.'K....{O.B)vt.

    Start Time: 1394623442
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
msg213250 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-03-12 11:38
Script to reproduce the issue:
---
import ssl
pem = ssl.get_server_certificate(('svn.python.org', 443), ca_certs="Lib/test/https_svn_python_org_root.pem")
print("PEM: %r" % pem)
---

It looks the handshake fails if like with PROTOCOL_SSLv3 which is the default protocol, but works with PROTOCOL_SSLv23.

_create_stdlib_context(), SSLContext and wrap_socket use PROTOCOL_SSLv23 which is said to be the "the most compatibility with other versions" protocol. Why get_server_certificate() uses PROTOCOL_SSLv3?

get_server_certificate() was added in 2007 by changeset 9041965a92f2 and it uses PROTOCOL_SSLv3 since this version.

"openssl s_client" says that the server speaks TLSv1.2 which is the most recent TLS version and probably the most secure. Is it possible somehow to try TLSv1.2, and then fallback to other versions if the latest version is not supported?

For the initial issue, it looks like a change at server side (svn.python.org), I don't think that ssl module, the unit test or the certificate of the authority changed recently. The python.org website has been changed recently.
msg213251 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-12 11:41
New changeset c13398566409 by Victor Stinner in branch 'default':
Issue #20896: Workaround the bug temporarely to fix buildbots
http://hg.python.org/cpython/rev/c13398566409
msg213294 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-03-12 20:50
The bug is not fixed, I commited a workaround. Please repon it.

Le mercredi 12 mars 2014, Benjamin Peterson <report@bugs.python.org> a
écrit :

>
> Changes by Benjamin Peterson <bp+pybugs@benjamin-peterson.org<javascript:;>
> >:
>
>
> ----------
> resolution:  -> fixed
> status: open -> closed
>
> _______________________________________
> Python tracker <report@bugs.python.org <javascript:;>>
> <http://bugs.python.org/issue20896>
> _______________________________________
>
msg213316 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-03-12 22:42
Ok, apparently the change was backed out and it was not needed (I can't make the test fail here, either). It would have been better to state it on the issue, though :-)
msg213318 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-03-12 22:43
That said, I agree it would be better to use "SSLv23" in get_server_certificate().
msg213323 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-12 23:11
New changeset 23add5382fb3 by Benjamin Peterson in branch '3.1':
use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes #20896)
http://hg.python.org/cpython/rev/23add5382fb3

New changeset 789ca594960f by Benjamin Peterson in branch '3.2':
use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes #20896)
http://hg.python.org/cpython/rev/789ca594960f

New changeset de97d0334314 by Benjamin Peterson in branch '2.7':
use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes #20896)
http://hg.python.org/cpython/rev/de97d0334314

New changeset 12df02358137 by Benjamin Peterson in branch '3.3':
merge 3.2 (#20896)
http://hg.python.org/cpython/rev/12df02358137

New changeset 0cba79667c7d by Benjamin Peterson in branch 'default':
merge 3.3 (#20896)
http://hg.python.org/cpython/rev/0cba79667c7d
msg213777 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-03-16 23:37
Benjamin: Could you please mention your change in Misc/NEWS?

Is it ok to change that in Python 3.1 & 3.2? Should the change be mentionned in the doc (:versionchanged:)?
msg213819 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-17 06:31
New changeset 9b91c23f071c by Benjamin Peterson in branch '3.4':
merge 3.3 (#20896)
http://hg.python.org/cpython/rev/9b91c23f071c
msg216502 - (view) Author: Roundup Robot (python-dev) Date: 2014-04-16 16:57
New changeset 55f62fa5bebc by Antoine Pitrou in branch 'default':
Issue #20896: ssl.get_server_certificate() now uses PROTOCOL_SSLv23, not PROTOCOL_SSLv3, for maximum compatibility.
http://hg.python.org/cpython/rev/55f62fa5bebc
msg216532 - (view) Author: (ddvento@ucar.edu) Date: 2014-04-16 18:06
This bug affected also the other versions I marked. Updating it, so people don't open duplicate bugs as I did with issue #21246
msg233521 - (view) Author: Roundup Robot (python-dev) Date: 2015-01-06 11:24
New changeset a8c4925e2359 by Victor Stinner in branch '3.4':
Issue #20896, #22935: The ssl.get_server_certificate() function now uses the
https://hg.python.org/cpython/rev/a8c4925e2359
History
Date User Action Args
2015-01-06 11:24:05python-devsetmessages: + msg233521
2014-04-16 18:06:02ddvento@ucar.edusetnosy: + ddvento@ucar.edu

messages: + msg216532
versions: + Python 3.1, Python 2.7, Python 3.2, Python 3.3
2014-04-16 16:58:56pitrousetstatus: open -> closed
resolution: fixed
2014-04-16 16:57:34python-devsetmessages: + msg216502
2014-04-16 16:50:21pitroulinkissue21246 superseder
2014-03-17 06:31:00python-devsetmessages: + msg213819
2014-03-16 23:37:06hayposetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg213777
2014-03-16 23:21:06GreenKeysetnosy: + GreenKey
2014-03-12 23:11:14python-devsetstatus: open -> closed
resolution: fixed
messages: + msg213323

stage: needs patch -> resolved
2014-03-12 22:43:24pitrousetversions: + Python 3.5
title: test_ssl.test_get_server_certificate() is failing: CA cert of svn.python.org changed? -> test_ssl.test_get_server_certificate() should use PROTOCOL_SSLv23, not PROTOCOL_SSLv3
messages: + msg213318

components: + Library (Lib)
type: behavior
stage: needs patch
2014-03-12 22:42:10pitrousetmessages: + msg213316
2014-03-12 22:36:07pitrousetstatus: closed -> open
resolution: fixed -> (no value)
2014-03-12 20:50:47hayposetmessages: + msg213294
2014-03-12 18:27:58benjamin.petersonsetstatus: open -> closed
resolution: fixed
2014-03-12 11:41:54python-devsetnosy: + python-dev
messages: + msg213251
2014-03-12 11:41:06koobssetnosy: + koobs
2014-03-12 11:38:46hayposetnosy: + christian.heimes
messages: + msg213250
2014-03-12 11:24:46hayposetmessages: + msg213249
2014-03-12 11:20:40haypocreate