classification
Title: PEP 466: upgrade OpenSSL in the Python 2.7 Windows builds
Type: enhancement Stage: resolved
Components: Build, Windows Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, loewis, ncoghlan, python-dev, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2014-05-09 12:12 by ncoghlan, last changed 2014-06-02 21:24 by zach.ware. This issue is now closed.

Files
File name Uploaded Description Edit
issue21462.diff zach.ware, 2014-05-09 15:28 review
224ca86e3919_backport.diff zach.ware, 2014-05-30 19:54 review
Messages (16)
msg218154 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-05-09 12:12
PEP 466 includes updating to a newer version of OpenSSL. This may be needed for the ssl module feature backports in issue 21308.
msg218171 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-05-09 15:28
Here's all the patch that should be necessary to do the upgrade.  Running test_ssl on 2.7 with 1.0.1g I do have a failure:

======================================================================
ERROR: test_socketserver (test.test_ssl.ThreadedTests)
Using a SocketServer to create and manage SSL connections.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "P:\ath\to\2.7\cpython\lib\test\test_ssl.py", line 1179, in test_socketserver
    f = urllib.urlopen(url)
  File "P:\ath\to\2.7\cpython\lib\urllib.py", line 87, in urlopen
    return opener.open(url)
  File "P:\ath\to\2.7\cpython\lib\urllib.py", line 208, in open
    return getattr(self, name)(url)
  File "P:\ath\to\2.7\cpython\lib\urllib.py", line 437, in open_https
    h.endheaders(data)
  File "P:\ath\to\2.7\cpython\lib\httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "P:\ath\to\2.7\cpython\lib\httplib.py", line 829, in _send_output
    self.send(msg)
  File "P:\ath\to\2.7\cpython\lib\httplib.py", line 791, in send
    self.connect()
  File "P:\ath\to\2.7\cpython\lib\httplib.py", line 1176, in connect
    self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
  File "P:\ath\to\2.7\cpython\lib\ssl.py", line 392, in wrap_socket
    ciphers=ciphers)
  File "P:\ath\to\2.7\cpython\lib\ssl.py", line 148, in __init__
    self.do_handshake()
  File "P:\ath\to\2.7\cpython\lib\ssl.py", line 310, in do_handshake
    self._sslobj.do_handshake()
IOError: [Errno socket error] [Errno 1] _ssl.c:510: error:140770FC:SSL routines:
SSL23_GET_SERVER_HELLO:unknown protocol

----------------------------------------------------------------------

But this looks closely related to a failure that I have on this machine using 3.4 (which is probably the fault of the way this network is set up):

======================================================================
ERROR: test_socketserver (test.test_ssl.ThreadedTests)
Using a SocketServer to create and manage SSL connections.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 1189, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 1128, in _send_request
    self.endheaders(body)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 1086, in endheaders
    self._send_output(message_body)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 924, in _send_output
    self.send(msg)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 859, in send
    self.connect()
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 1221, in connect
    super().connect()
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 839, in connect
    self._tunnel()
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 822, in _tunnel
    message.strip()))
OSError: Tunnel connection failed: 403 Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "P:\ath\to\3.4\cpython\lib\test\test_ssl.py",line 2315, in test_socketserver
    f = urllib.request.urlopen(url)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 455, in open
    response = self._open(req, data)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 473, in _open
    '_open', req)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 433, in _call_chain
    result = func(*args)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 1230, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 1192, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error Tunnel connection failed: 403 Forbidden>

----------------------------------------------------------------------

As such, I'll leave it to someone else to properly test this before committing.
msg218176 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2014-05-09 16:35
I just tried it and had no trouble building and running the ssl tests on Windows.

> python Lib\test\regrtest.py -u network -v test_ssl
> ...
> Ran 38 tests in 7.700s
> 
> OK (skipped=2)
msg218180 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-05-09 17:16
Thanks, Steve.  Nick, I assume 1.0.1g is the target version?
msg218193 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2014-05-09 18:55
Builds & tests ok here on a fresh checkout (of cpython & openssl-1.0.1g)
msg218315 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-05-12 06:25
Yes, since OpenSSL 1.0.2 is still in beta, the target version for 2.7.7 would be 1.0.1g
msg219359 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-05-29 18:59
Steve or Tim, did one of you want to commit this since you can run a clean test?
msg219396 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2014-05-30 15:52
I can commit it, though I don't know how it'll affect Benjamin's release branch?

(Obviously the build will be fine either way - I had the patch applied for 2.7.7rc1.)
msg219403 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-05-30 16:49
Go ahead and commit; it will be up to Benjamin to cherry-pick it to his release branch (or to ask you to do it).
msg219407 - (view) Author: Roundup Robot (python-dev) Date: 2014-05-30 18:02
New changeset f6e47d27f67a by Steve Dower in branch '2.7':
Issue #21462 PEP 466: upgrade OpenSSL in the Python 2.7 Windows builds
http://hg.python.org/cpython/rev/f6e47d27f67a
msg219422 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-05-30 19:54
Looks like the AMD64 buildbot isn't happy with OpenSSL 1.0.1g: http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%202.7/builds/1223/steps/compile/logs/stdio

Here's my best stab-in-the-dark guess at fixing what's wrong; I don't have the ability to build x64 2.7 handy.
msg219463 - (view) Author: Roundup Robot (python-dev) Date: 2014-05-31 18:01
New changeset e90024db8d6c by Benjamin Peterson in branch '2.7':
openssl requires nasm (#21462)
http://hg.python.org/cpython/rev/e90024db8d6c
msg219464 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-05-31 18:11
On Fri, May 30, 2014, at 12:54, Zachary Ware wrote:
> 
> Zachary Ware added the comment:
> 
> Looks like the AMD64 buildbot isn't happy with OpenSSL 1.0.1g:
> http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%202.7/builds/1223/steps/compile/logs/stdio
> 
> Here's my best stab-in-the-dark guess at fixing what's wrong; I don't
> have the ability to build x64 2.7 handy.

Good call. That seems to have satisfied the AMD64 bot (at least in the
compile stage).
msg219465 - (view) Author: Roundup Robot (python-dev) Date: 2014-05-31 18:24
New changeset 7095a9bfbe76 by Steve Dower in branch '2.7':
Issue #21462 PEP 466: upgrade OpenSSL in the Python 2.7 Windows builds
http://hg.python.org/cpython/rev/7095a9bfbe76

New changeset 8e2cda42b466 by Benjamin Peterson in branch '2.7':
openssl requires nasm (#21462)
http://hg.python.org/cpython/rev/8e2cda42b466
msg219466 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-05-31 18:30
Will this be in 2.7.7? If so, I'll update the draft What's New changes in issue #21569 accordingly (as well as the implementation status in PEP 466)
msg219467 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-05-31 18:31
Yes.

On Sat, May 31, 2014, at 11:30, Nick Coghlan wrote:
> 
> Nick Coghlan added the comment:
> 
> Will this be in 2.7.7? If so, I'll update the draft What's New changes in
> issue #21569 accordingly (as well as the implementation status in PEP
> 466)
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue21462>
> _______________________________________
History
Date User Action Args
2014-06-02 21:24:03zach.waresetstage: resolved
2014-05-31 18:31:39benjamin.petersonsetstatus: open -> closed
resolution: fixed
2014-05-31 18:31:16benjamin.petersonsetmessages: + msg219467
2014-05-31 18:30:35ncoghlansetmessages: + msg219466
2014-05-31 18:24:33python-devsetmessages: + msg219465
2014-05-31 18:11:59benjamin.petersonsetmessages: + msg219464
2014-05-31 18:01:48python-devsetmessages: + msg219463
2014-05-30 19:54:39zach.waresetfiles: + 224ca86e3919_backport.diff

messages: + msg219422
2014-05-30 18:02:44python-devsetnosy: + python-dev
messages: + msg219407
2014-05-30 16:49:08zach.waresetmessages: + msg219403
2014-05-30 15:52:31steve.dowersetmessages: + msg219396
2014-05-29 18:59:05zach.waresetmessages: + msg219359
2014-05-12 06:25:07ncoghlansetnosy: + benjamin.peterson
messages: + msg218315
2014-05-09 18:55:42tim.goldensetnosy: + tim.golden
messages: + msg218193
2014-05-09 17:16:30zach.waresetmessages: + msg218180
2014-05-09 16:35:10steve.dowersetmessages: + msg218176
2014-05-09 15:28:28zach.waresetfiles: + issue21462.diff

nosy: + zach.ware
messages: + msg218171

keywords: + patch
2014-05-09 12:12:37ncoghlansetcomponents: + Build
title: PEP 466: upgrade OpenSSL -> PEP 466: upgrade OpenSSL in the Python 2.7 Windows builds
2014-05-09 12:12:07ncoghlancreate