classification
Title: test_ssl fails with openssl 1.1.0f: test_alpn_protocols()
Type: behavior Stage: patch review
Components: Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: benjamin.peterson, christian.heimes, cstratak, gregory.p.smith, haypo, larry, ned.deily, r.david.murray
Priority: normal Keywords:

Created on 2017-06-20 15:26 by cstratak, last changed 2017-07-24 17:05 by r.david.murray.

Files
File name Uploaded Description Edit
build.log cstratak, 2017-06-20 15:26
Pull Requests
URL Status Linked Edit
PR 2305 open christian.heimes, 2017-06-20 16:35
Messages (13)
msg296456 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2017-06-20 15:26
After updating openssl in Fedora 26 from 1.1.0e to 1.1.0f the test_alpn_protocols from test_ssl started failing:

======================================================================
FAIL: test_alpn_protocols (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.6.1/Lib/test/test_ssl.py", line 3261, in test_alpn_protocols
    self.assertIsInstance(stats, ssl.SSLError)
AssertionError: {'compression': None, 'cipher': ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), 'peercert': {}, 'client_alpn_protocol': None, 'client_npn_protocol': None, 'version': 'TLSv1.2', 'session_reused': False, 'session': <_ssl.Session object at 0x7f846ed97740>, 'server_alpn_protocols': [None], 'server_npn_protocols': [None], 'server_shared_ciphers': [[('ECDHE-ECDSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('ECDHE-ECDSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('ECDHE-RSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('DHE-DSS-AES256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), ('DHE-DSS-AES128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CHACHA20-POLY1305', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-CCM8', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-CCM', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-AES256-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-AES256-SHA', 'TLSv1.0', 256), ('ECDHE-RSA-AES256-SHA', 'TLSv1.0', 256), ('DHE-RSA-AES256-CCM8', 'TLSv1.2', 256), ('DHE-RSA-AES256-CCM', 'TLSv1.2', 256), ('DHE-RSA-AES256-SHA256', 'TLSv1.2', 256), ('DHE-DSS-AES256-SHA256', 'TLSv1.2', 256), ('DHE-RSA-AES256-SHA', 'SSLv3', 256), ('DHE-DSS-AES256-SHA', 'SSLv3', 256), ('ECDHE-ECDSA-AES128-CCM8', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-CCM', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-AES128-SHA256', 'TLSv1.2', 128), ('ECDHE-ECDSA-AES128-SHA', 'TLSv1.0', 128), ('ECDHE-RSA-AES128-SHA', 'TLSv1.0', 128), ('DHE-RSA-AES128-CCM8', 'TLSv1.2', 128), ('DHE-RSA-AES128-CCM', 'TLSv1.2', 128), ('DHE-RSA-AES128-SHA256', 'TLSv1.2', 128), ('DHE-DSS-AES128-SHA256', 'TLSv1.2', 128), ('DHE-RSA-AES128-SHA', 'SSLv3', 128), ('DHE-DSS-AES128-SHA', 'SSLv3', 128), ('ECDHE-ECDSA-CAMELLIA256-SHA384', 'TLSv1.2', 256), ('ECDHE-RSA-CAMELLIA256-SHA384', 'TLSv1.2', 256), ('ECDHE-ECDSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('ECDHE-RSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CAMELLIA256-SHA256', 'TLSv1.2', 256), ('DHE-DSS-CAMELLIA256-SHA256', 'TLSv1.2', 256), ('DHE-RSA-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-DSS-CAMELLIA128-SHA256', 'TLSv1.2', 128), ('DHE-RSA-CAMELLIA256-SHA', 'SSLv3', 256), ('DHE-DSS-CAMELLIA256-SHA', 'SSLv3', 256), ('DHE-RSA-CAMELLIA128-SHA', 'SSLv3', 128), ('DHE-DSS-CAMELLIA128-SHA', 'SSLv3', 128), ('AES256-GCM-SHA384', 'TLSv1.2', 256), ('AES128-GCM-SHA256', 'TLSv1.2', 128), ('AES256-CCM8', 'TLSv1.2', 256), ('AES256-CCM', 'TLSv1.2', 256), ('AES128-CCM8', 'TLSv1.2', 128), ('AES128-CCM', 'TLSv1.2', 128), ('AES256-SHA256', 'TLSv1.2', 256), ('AES128-SHA256', 'TLSv1.2', 128), ('AES256-SHA', 'SSLv3', 256), ('AES128-SHA', 'SSLv3', 128), ('CAMELLIA256-SHA256', 'TLSv1.2', 256), ('CAMELLIA128-SHA256', 'TLSv1.2', 128), ('CAMELLIA256-SHA', 'SSLv3', 256), ('CAMELLIA128-SHA', 'SSLv3', 128)]]} is not an instance of <class 'ssl.SSLError'>

Full build log attached
msg296458 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2017-06-20 15:29
Note: Python version is 3.6.1
msg296465 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-20 15:54
The ALPN test expects an error on OpenSSL >= 1.1, and an error on older OpenSSL versions.

Note: I don't know what is ALPN :-) I found:
https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation
msg296468 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-06-20 16:19
I can confirm that OpenSSL has changed behavior of ALPN hook between 1.1.0e and 1.1.0f. The change was probably introduced by https://github.com/openssl/openssl/pull/3158/commits/b3159f23b293c3d1870ab7b816e4e07386efbe53 I need to investigate further.
msg296470 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-06-20 16:37
Ned, I like to address this issue for 3.6.2. The fix only affects one test and documentation.
msg297458 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-07-01 00:27
Sorry for the delay.  It's clear this needs to get fixed so there's no need to wait to merge PRs into 3.6, 3.5, and 2.7.  If the PR gets merged into 3.6 soon, I'll pull it into 3.6.2 as well.
msg297872 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-07-07 09:03
Sorry, this didn't make it in time for 3.6.2.  There is still at least a couple of weeks to get it into 3.5.4 and 2.7.14.
msg298090 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-07-10 21:46
The test now fails on AMD64 Debian PGO 3.x:

http://buildbot.python.org/all/builders/AMD64%20Debian%20PGO%203.x/builds/985/steps/test/logs/stdio

FAIL: test_alpn_protocols (test.test_ssl.ThreadedTests)
msg298120 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-07-11 05:02
I updated to PGO buildbot from Debian 8 "jessie" to Debian 9 "stretch" which revealed the failures.
msg298911 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2017-07-23 20:45
I don't see how a fix for a *test* can be considered a *release blocker*.  The PR literally doesn't change Python's behavior; it only modifies two text files and a test.  There is no crash or exploitable security hole being addressed here.
msg298912 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2017-07-23 20:46
Quoting from the Python Dev Guide:

"As a guideline, critical and above are usually reserved for crashes, serious regressions or breakage of very important APIs. Whether a bug is a release blocker is a decision better left to the release manager so, in any doubt, add him or her to the nosy list."
msg298944 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-07-24 09:22
Test still fails. Failure on AMD64 Debian PGO 2.7:

http://buildbot.python.org/all/builders/AMD64%20Debian%20PGO%202.7/builds/243/steps/test/logs/stdio

======================================================================
FAIL: test_alpn_protocols (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/slaves/enable-optimizations-bot/2.7.gps-debian-profile-opt.nondebug/build/Lib/test/test_ssl.py", line 2971, in test_alpn_protocols
    self.assertIsInstance(stats, ssl.SSLError)
AssertionError: {'compression': None, 'client_npn_protocol': None, 'cipher': ('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1.2', 256), 'peercert': {}, 'server_npn_protocols': [None], 'client_alpn_protocol': None, 'version': u'TLSv1.2', 'server_alpn_protocols': [None]} is not an instance of <class 'ssl.SSLError'>
msg298994 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-07-24 17:05
Well, the reason one *might* consider a test failure as a release blocker (and I'm not saying you should, I'm just explaining the possible logic) is that distros would understandably like the test suite to pass before they include a release in their distribution.
History
Date User Action Args
2017-07-24 17:05:07r.david.murraysetnosy: + r.david.murray
messages: + msg298994
2017-07-24 09:22:50hayposetmessages: + msg298944
2017-07-24 09:22:21hayposettitle: test_ssl fails with openssl 1.1.0f -> test_ssl fails with openssl 1.1.0f: test_alpn_protocols()
2017-07-23 20:46:33larrysetmessages: + msg298912
2017-07-23 20:45:21larrysetpriority: release blocker -> normal

messages: + msg298911
2017-07-12 23:50:43ned.deilylinkissue30914 superseder
2017-07-11 05:02:51gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg298120
2017-07-10 21:46:12hayposetmessages: + msg298090
2017-07-07 09:03:04ned.deilysetmessages: + msg297872
2017-07-01 00:27:06ned.deilysetmessages: + msg297458
2017-06-20 19:40:12christian.heimessetassignee: christian.heimes
type: behavior
stage: patch review
2017-06-20 16:37:06christian.heimessetpriority: normal -> release blocker
versions: + Python 2.7, Python 3.5, Python 3.7
nosy: + ned.deily, benjamin.peterson, larry

messages: + msg296470
2017-06-20 16:35:58christian.heimessetpull_requests: + pull_request2352
2017-06-20 16:19:46christian.heimessetmessages: + msg296468
2017-06-20 15:54:51hayposetmessages: + msg296465
2017-06-20 15:29:36cstrataksetmessages: + msg296458
versions: + Python 3.6
2017-06-20 15:27:53hayposetnosy: + haypo, christian.heimes
2017-06-20 15:26:41cstratakcreate