Title: Cannot tunnel TLS connection through TLS connection
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.7, Python 3.6, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Maximilian Blochberger, christian.heimes, docs@python, terry.reedy
Priority: normal Keywords:

Created on 2017-01-31 07:19 by Maximilian Blochberger, last changed 2017-09-06 19:57 by christian.heimes.

Messages (6)
msg286517 - (view) Author: Maximilian Blochberger (Maximilian Blochberger) Date: 2017-01-31 07:19
I have the following scenario: Client → Proxy → Target.

The following two scenarios are working perfectly fine:

1) Establishing a TLS-secured connection to the proxy and then tunnel traffic through that connection to the target. This results in the proxy being able to observe and manipulate the traffic in both directions. It protects against an adversary who has no control over the proxy, e.g. it prevents observers from learning that you are using a proxy (if the IP/port is not known) and from reading the actual traffic.

2) Establish a non-secured connection to the proxy and then tunnel TLS-secured traffic through that connection to the target. That prevents the proxy from being able to observe or manipulate the traffic. Although an observer could learn that you are using a proxy and what target you are connecting to.

Now what I tried was to establish a TLS-secured connection to the proxy and then to establish a TLS-secured tunnel to the target, effectively resulting in two layers of TLS in between the client and the proxy. This would protect from an observer learning that you are using a proxy and where you connect to (the proxy still knows) but preventing the proxy from observing and manipulating the actual traffic to the target.

This does not work in Python 3.6. The TLS-secured connection to the proxy is straight forward and can be easily done with ssl.SSLContext.wrap_socket(). The TCP connection between the proxy and the target can then be established by issuing an HTTP CONNECT request. The response can than be read without closing the connection as done in http.client.HTTPConnection._tunnel(). Now my idea was to call ssl.SSLContext.wrap_socket() again (with a different context for the target) and send traffic through that. Unfortunately the TLS handshake fails with the error message "unknown protocol". I looked into the actual traffic transmitted and realised that the handshake was performed in plain text – so effectively stripping the TLS layer that was established already – which results in the proxy server not knowing how to handle the traffic (as it is not TLS-secured) aborting the connection (and reporting a fatal TLS alert).

This leads to the conclusion that another call to ssl.SSLContext.wrap_socket() will override a previous call of the same function (different context object though). I think this is unexpected behaviour.

It might be easier to handle such scenarios if a tunnel would be a separate http.client.HTTP(S)Connection object, see issue #24964. This would also allow to handle ssl-specific calls such as ssl.SSLSocket.getpeercert() as each layer probably uses different certificates.
msg286519 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-01-31 09:23
You cannot use wrap_socket() to wrap a SSLSocket into another SSLSocket. Python uses SSL_set_fd() to wrap the socket's file descriptor. OpenSSL directly pulls and pushes data to the connection. Instead you have to use wrap_bio() and do I/O on your own.
msg286787 - (view) Author: Maximilian Blochberger (Maximilian Blochberger) Date: 2017-02-02 13:37
Okay, I see, thanks for the hint. That worked perfectly – I found `asyncio.sslproto._SSLPipe` very useful for that purpose.

I personally consider the behaviour of `ssl.SSLContext.wrap_socket()` unexpected and would raise an exception if that method call is tried on an instance of `ssl.SSLSocket`. But as this would be a change that could lead to backwards compatibility issues (if developers depend on that behaviour) this is probably not a good idea.

I think that the documentation for `ssl.SSLContext.wrap_socket()` has this behaviour to avoid future confusion – including a hint to make use of `ssl.SSLContext.wrap_bio()` instead.
msg286897 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-02-03 22:34
Maximilian, I could not tell if you are still requesting that something be changed, or if this should be closed.
msg286928 - (view) Author: Maximilian Blochberger (Maximilian Blochberger) Date: 2017-02-04 08:29
Yes. There should be at least an explanation of this behaviour in the documentation of the wrap_socket() function.

I would additionally raise an exception if wrap_socket() is called and a socket is passed that is already wrapped. But I'm not sure if that is considered as an acceptable choice, as I am unfamiliar with Python development.
msg301507 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-06 19:57
A documentation update wouldn't hurt, though.
Date User Action Args
2017-09-06 19:57:26christian.heimessetversions: - Python 3.5
nosy: + docs@python

messages: + msg301507

assignee: christian.heimes -> docs@python
components: + Documentation, - SSL
2017-02-21 04:30:22martin.panterlinkissue29610 superseder
2017-02-21 04:29:41martin.pantersetstage: needs patch
versions: + Python 2.7, Python 3.5, Python 3.7
2017-02-04 08:29:42Maximilian Blochbergersetmessages: + msg286928
2017-02-03 22:34:19terry.reedysetnosy: + terry.reedy
messages: + msg286897
2017-02-02 13:37:47Maximilian Blochbergersetmessages: + msg286787
2017-01-31 09:23:43christian.heimessetmessages: + msg286519
2017-01-31 07:19:43Maximilian Blochbergercreate