classification
Title: Debian Sid/Buster: Cannot enable TLS 1.0/1.1 with PROTOCOL_TLS
Type: behavior Stage: patch review
Components: SSL Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Alex Gaynor, adrianv, christian.heimes, doko, ned.deily
Priority: deferred blocker Keywords: patch

Created on 2017-09-13 15:48 by adrianv, last changed 2018-01-28 21:51 by christian.heimes.

Files
File name Uploaded Description Edit
poc.py adrianv, 2017-09-13 15:48
Pull Requests
URL Status Linked Edit
PR 3662 open christian.heimes, 2017-09-19 19:09
Messages (17)
msg302081 - (view) Author: Adrian Vollmer (adrianv) Date: 2017-09-13 15:48
According to the documentation (https://docs.python.org/2/library/ssl.html#ssl.PROTOCOL_TLS), using ssl_version = ssl.PROTOCOL_TLS in a server socket should offer all TLS/SSL versions. However, it only offers TLSv1_2.

I attached a proof of concept.


$ python3 poc.py
3.5.4 (default, Aug 12 2017, 14:08:14)
[GCC 7.1.0]
OpenSSL 1.1.0f  25 May 2017
[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:719)
[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:719)
b'test\n'

$ python2 poc.py
2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118]
OpenSSL 1.1.0f  25 May 2017
[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:661)
[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:661)
test


To connect with s_client:

 $ for i in {tls1,tls1_1,tls1_2} ; do echo test | openssl s_client -connect localhost:3333 -CAfile server.pem -quiet -$i ; done
140164347663616:error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version:../ssl/record/rec_layer_s3.c:1399:SSL alert number 70
139926441944320:error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version:../ssl/record/rec_layer_s3.c:1399:SSL alert number 70
depth=0 C = AU, ST = Some-State, O = Internet Widgits Pty Ltd
verify return:1
read:errno=0
msg302082 - (view) Author: Alex Gaynor (Alex Gaynor) Date: 2017-09-13 15:49
What operating system are you on?
msg302084 - (view) Author: Adrian Vollmer (adrianv) Date: 2017-09-13 15:53
Debian buster/sid
msg302085 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-13 16:03
Debian Buster has patched OpenSSL to disable TLS 1.0 and 1.1 by default, 
https://lists.debian.org/debian-devel-announce/2017/08/msg00004.html
msg302086 - (view) Author: Adrian Vollmer (adrianv) Date: 2017-09-13 16:10
I read about that, but I don't understand. If I use openssl s_server -port 3333  , I can connect using either one of the three protocols.

Even if that's the new default, is there no way now to get python on Buster/Sid to use OpenSSL in a non-default mode and have it offer all three versions?
msg302088 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-13 16:16
You have to enable the protocols by applying a reverse bitmask to SSLContext.options:

ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
ctx.load_cert_chain('server.pem')
ctx.options &= ~(ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1)

sslsock = ctx.wrap_socket(s, server_side=True)
msg302091 - (view) Author: Adrian Vollmer (adrianv) Date: 2017-09-13 16:33
Doesn't seem to do anything:

>>> ctx.options
2181170175L
>>> ctx.options & ~(ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1)
2181170175L
msg302092 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-13 16:37
Please report this issue to the Debian maintainers. I don't know how Debian has disabled TLS 1.0 and TLS 1.1 for the SSL_METHOD *TLS_method(void). It might not be possible to enable auto-negotiation for old protocols at all.
msg302093 - (view) Author: Adrian Vollmer (adrianv) Date: 2017-09-13 16:41
Okay, thanks for your time!
msg302094 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-13 16:41
Ah, here we go: https://anonscm.debian.org/viewvc/pkg-openssl/openssl/branches/1.1.0/debian/patches/tls1_2_default.patch

Debian patched the default for SSL_CTX_set_min_proto_version(). The SSL_CTX_set_min_proto_version() and SSL_CTX_set_max_proto_version() API calls are OpenSSL 1.1.0-only and not available from Python. It is not possible to override the minimum version from Python.
msg302097 - (view) Author: Adrian Vollmer (adrianv) Date: 2017-09-13 17:07
I have a workaround for now:

        versions = [ssl.PROTOCOL_TLSv1,
                    ssl.PROTOCOL_TLSv1_1,
                    ssl.PROTOCOL_TLSv1_2,
                   ]
        firstbytes = s.recv(16, socket.MSG_PEEK)
        ss = ssl.wrap_socket(
            s,
            server_side=True,
            certfile="server.pem",
            keyfile="server.pem",
            #  ssl_version=versions[ord(firstbytes[10])-1] # python2
            ssl_version=versions[firstbytes[10]-1]
        )

How much of an ugly hack is this? :)
msg302099 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-13 17:10
It's an ugly hack and not a long term solution. The PROTOCOL_TLSv* constants and ssl.wrap_socket() are discouraged and will be removed soon.
msg302100 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-13 17:11
Matthias, this issue affects Debian and probably Ubuntu, too. Could you please discuss it with Debian maintainers and propose a workaround? Python does not expose the new OpenSSL 1.1.0 SSL_CTX_set_min_proto_version() and SSL_CTX_set_max_proto_version() calls. We only support SSL_CTX_set_options() with SSL_OP_NO_TLSv1 and SSL_OP_NO_TLSv1_1.
msg302561 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-19 19:19
PR 3662 undos Debian's patching of OpenSSL. I'm not keen to undo a security improvement. However Debian is breaking backwards compatibility. For Python 3.7 we could consider to disable TLS 1.0 and TLS 1.1 for PROTOCOL_TLS_SERVER and PROTOCOL_TLS_CLIENT.
msg307105 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-11-28 01:47
FWIW, Debian seems to have re-enabled TLS 1.0 and 1.1 in "testing".  As a result, test_ssl now passes again.

openssl (1.1.0g-1) unstable; urgency=medium

  * New upstream version
    - Fixes CVE-2017-3735
    - Fixes CVE-2017-3736
  * Remove patches applied upstream
  * Temporary enable TLS 1.0 and 1.1 again (#875423)
  * Attempt to fix testsuite race condition
  * update no-symbolic.patch to apply

 -- Kurt Roeckx <kurt@roeckx.be>  Thu, 02 Nov 2017 15:22:48 +0100
msg310996 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-01-28 21:11
I'm not clear if this is still needed, i.e. has Debian backed off on their change across the board?  If it is still needed, I'm going to allow an extension for landing of it until 3.7.0b2, currently scheduled for 2018-02-26.  If anyone else can help Christian get this in before b2, that would be great.  I'm removing older versions for now. We can discuss potential backports after the code lands.
msg311008 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-28 21:51
The feature is still useful -- whether or not Debian disables TLS 1.0 and 1.1. The new API is easier to use and more convenient than the old bitmask approach. "option &=~OP_NO_SSLv3" is just horrible. :)
History
Date User Action Args
2018-01-28 21:51:03christian.heimessetmessages: + msg311008
2018-01-28 21:11:30ned.deilysetpriority: normal -> deferred blocker

messages: + msg310996
versions: + Python 3.8, - Python 2.7, Python 3.6
2017-11-28 01:47:09ned.deilysetnosy: + ned.deily
messages: + msg307105
2017-09-19 19:19:36christian.heimessetmessages: + msg302561
2017-09-19 19:09:33christian.heimessetkeywords: + patch
stage: patch review
pull_requests: + pull_request3651
2017-09-13 17:11:20christian.heimessetversions: + Python 3.6, Python 3.7, - Python 3.5
nosy: + doko

messages: + msg302100

assignee: christian.heimes ->
2017-09-13 17:10:59christian.heimessetmessages: + msg302099
2017-09-13 17:07:03adrianvsetmessages: + msg302097
versions: - Python 3.6, Python 3.7
2017-09-13 16:47:43christian.heimessetversions: + Python 3.6, Python 3.7
2017-09-13 16:47:34christian.heimessettitle: ssl.PROTOCOL_TLS only select TLSv1.2 -> Debian Sid/Buster: Cannot enable TLS 1.0/1.1 with PROTOCOL_TLS
2017-09-13 16:41:31christian.heimessetmessages: + msg302094
2017-09-13 16:41:27adrianvsetmessages: + msg302093
2017-09-13 16:37:54christian.heimessetmessages: + msg302092
2017-09-13 16:33:03adrianvsetmessages: + msg302091
2017-09-13 16:16:33christian.heimessetmessages: + msg302088
2017-09-13 16:10:07adrianvsetmessages: + msg302086
2017-09-13 16:03:42christian.heimessetmessages: + msg302085
2017-09-13 15:53:55adrianvsetmessages: + msg302084
2017-09-13 15:49:39Alex Gaynorsetnosy: + Alex Gaynor
messages: + msg302082
2017-09-13 15:48:25adrianvcreate