classification
Title: Support for the NPN extension to TLS/SSL
Type: enhancement Stage: committed/rejected
Components: Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: colinmarc, jcea, loewis, marcelo_fernandez, pitrou, python-dev, ssm
Priority: normal Keywords: patch

Created on 2012-03-05 20:21 by colinmarc, last changed 2012-05-02 19:31 by colinmarc. This issue is now closed.

Files
File name Uploaded Description Edit
npn_patch.diff colinmarc, 2012-03-05 20:20 review
npn_patch_py3.diff colinmarc, 2012-03-10 17:34 review
npn_patch_py3.diff colinmarc, 2012-03-10 18:21 review
npn_openssl_ref.c colinmarc, 2012-03-10 20:22 relevant openssl code
npn_patch_py3.diff colinmarc, 2012-03-11 19:19 review
npn_patch_py3.diff colinmarc, 2012-03-12 21:00 review
npn.patch pitrou, 2012-03-17 20:01 review
Messages (18)
msg154973 - (view) Author: Colin Marc (colinmarc) Date: 2012-03-05 20:20
Recent versions of OpenSSL (1.0.1 and greater) support a new extension to SSL/TLS called Next Protocol Negotiation, defined here: http://tools.ietf.org/html/draft-agl-tls-nextprotoneg-02. 

The extension allows servers and clients to advertise which protocols they support (for example, both HTTP and SPDY) and then agree on one during the handshake according to a simple algorithm.

This patch to 2.7 adds support for the NPN extension via another parameter to ssl.wrap_socket, called 'npn_protocols', and by using the OpenSSL API. It should fail gracefully if the linked version of OpenSSL has no support for NPN, using a macro guard. Once the handshake is completed, SSLSocket.selected_protocol() returns whatever was agreed upon.

Although I included client/server tests with the patch, testing this functionality in real-life situations proved difficult. Google chrome has SPDY and NPN functionality baked in, so I wrote a simple socket server that advertises SPDY/2 in addition to HTTP/1.1. Chrome, pointed at this server, correctly completed the handshake and started merrily sending SPDY control frames.
msg154978 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-03-05 21:23
There is zero chance that this can go into 2.7. So if you want to see it included, please port it to Python 3, and it may become part of Python 3.3 or 3.4.
msg154979 - (view) Author: Colin Marc (colinmarc) Date: 2012-03-05 21:30
If I ported it to 3.3 or 3.4, would it then be backported to 2.7? Or is there zero chance of that either? If so, why? I apologize, I'm new to the process.
msg154980 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-03-05 21:39
> If I ported it to 3.3 or 3.4, would it then be backported to 2.7? Or
> is there zero chance of that either? If so, why? I apologize, I'm new
> to the process.

It won't be backported. Python 2.7 is in bug-fix mode; no new features
are allowed it it. In addition, there won't be another 2.x release
(see PEP 404), so new features can only be added to Python 3.

If this means that you'll lose interest in this issue - that's fine.
Let us know whether you then would rather withdraw the patch, or
leave it open in case someone is motivated to port it. In the latter
case, please submit a contributor's form to the PSF.
msg154982 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-05 21:54
Hello Marc,

> Recent versions of OpenSSL (1.0.1 and greater) support a new extension 
> to SSL/TLS called Next Protocol Negotiation, defined here:
> http://tools.ietf.org/html/draft-agl-tls-nextprotoneg-02. 

Apparently this is an IETF draft. Do you know if it is stabilized enough that it won't change significantly?

Also, please notice that the ssl module (starting from Python 3.2) now exposes the notion of an SSL context. The setting of NPN parameters should probably be exposed as a context method and/or a parameter to SSLContext.wrap_socket().
(see http://docs.python.org/dev/library/ssl.html#ssl-contexts for docs)
msg154983 - (view) Author: Colin Marc (colinmarc) Date: 2012-03-05 22:10
Re the IETF draft: I'm not sure. However, I didn't actually have to implement the specification at all - that was all handled by OpenSSL. My patch just calls the appropriate SSL_CTX_* methods. 

Thanks for the tip. I'm still interested in this getting included, so I'll work on porting it over.
msg155326 - (view) Author: Colin Marc (colinmarc) Date: 2012-03-10 17:34
Here's an updated patch against 3.3.
msg155335 - (view) Author: Colin Marc (colinmarc) Date: 2012-03-10 18:21
Oops, I had my vim configured wrong and left a few tab characters in there. Here's another updated patch =)
msg155350 - (view) Author: Colin Marc (colinmarc) Date: 2012-03-10 20:22
Here's the OpenSSL code I referenced for my implementation. It's an excerpt of ssl/lib_ssl.c, starting at line 1514.
msg155408 - (view) Author: Colin Marc (colinmarc) Date: 2012-03-11 19:19
Updated patch.
msg155475 - (view) Author: Colin Marc (colinmarc) Date: 2012-03-12 21:00
More updates to the patch.
msg156193 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-17 19:38
Sorry for the delay. I've run the tests (with OpenSSL 1.0.1-beta3) in debug mode and got an error:

======================================================================
ERROR: test_npn_ext (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_ssl.py", line 1882, in test_npn_ext
    chatty=True, connectionchatty=True)
  File "/home/antoine/cpython/default/Lib/test/test_ssl.py", line 1210, in server_params_test
    s.connect((HOST, server.port))
  File "/home/antoine/cpython/default/Lib/ssl.py", line 543, in connect
    self._real_connect(addr, False)
  File "/home/antoine/cpython/default/Lib/ssl.py", line 533, in _real_connect
    self.do_handshake()
  File "/home/antoine/cpython/default/Lib/ssl.py", line 513, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:434: error:140920E3:SSL routines:SSL3_GET_SERVER_HELLO:parse tlsext


I've determined that this is because of the use of strlen() on a non-zero terminated string. I'll try to come up with an updated patch.
msg156198 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-17 20:01
Here is a fixed patch. It also came to me that "selected_protocol" could be ambiguous, so I renamed it to "selected_npn_protocol".
msg156525 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-21 23:34
New changeset 2514a4e2b3ce by Antoine Pitrou in branch 'default':
Issue #14204: The ssl module now has support for the Next Protocol Negotiation extension, if available in the underlying OpenSSL library.
http://hg.python.org/cpython/rev/2514a4e2b3ce
msg156528 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-03-22 01:04
Closing since the buildbots don't seem to show any new failures after the commit. Thank you for your contribution!
msg159815 - (view) Author: Colin Marc (colinmarc) Date: 2012-05-02 19:29
Just noticed this is missing from "What's new in Python 3.3": http://docs.python.org/dev/whatsnew/3.3.html. 

Should I submit a patch for that?
msg159816 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-05-02 19:30
> Just noticed this is missing from "What's new in Python 3.3": http://docs.python.org/dev/whatsnew/3.3.html. 

> Should I submit a patch for that?

No need for that, the What's New document usually gets filled later in the release cycle.
msg159817 - (view) Author: Colin Marc (colinmarc) Date: 2012-05-02 19:31
Ah ok, just curious. Thanks!
History
Date User Action Args
2012-05-02 19:31:37colinmarcsetmessages: + msg159817
2012-05-02 19:30:26pitrousetmessages: + msg159816
2012-05-02 19:29:39colinmarcsetmessages: + msg159815
2012-03-29 03:05:38marcelo_fernandezsetnosy: + marcelo_fernandez
2012-03-22 01:05:00pitrousetstatus: open -> closed
resolution: fixed
messages: + msg156528

stage: committed/rejected
2012-03-21 23:34:41python-devsetnosy: + python-dev
messages: + msg156525
2012-03-17 20:01:49pitrousetfiles: + npn.patch

messages: + msg156198
2012-03-17 19:38:17pitrousetmessages: + msg156193
2012-03-12 21:00:21colinmarcsetfiles: + npn_patch_py3.diff

messages: + msg155475
2012-03-12 13:58:57jceasetnosy: + jcea
2012-03-11 19:19:23colinmarcsetfiles: + npn_patch_py3.diff

messages: + msg155408
2012-03-10 20:22:06colinmarcsetfiles: + npn_openssl_ref.c

messages: + msg155350
2012-03-10 18:28:08ssmsetnosy: + ssm
2012-03-10 18:21:34colinmarcsetfiles: + npn_patch_py3.diff

messages: + msg155335
2012-03-10 17:34:36colinmarcsetfiles: + npn_patch_py3.diff

messages: + msg155326
2012-03-05 22:10:09colinmarcsetmessages: + msg154983
2012-03-05 21:54:27pitrousetnosy: + pitrou
messages: + msg154982
2012-03-05 21:39:40loewissetmessages: + msg154980
2012-03-05 21:30:23colinmarcsetmessages: + msg154979
2012-03-05 21:23:07loewissetnosy: + loewis

messages: + msg154978
versions: + Python 3.3, - Python 2.7
2012-03-05 20:21:01colinmarccreate