classification
Title: Let OpenSSL verify hostname and IP address
Type: enhancement Stage: resolved
Components: SSL Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: Socob, christian.heimes, jwilk, vstinner
Priority: high Keywords: patch

Created on 2017-09-08 21:04 by christian.heimes, last changed 2018-02-26 08:41 by christian.heimes. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3462 merged christian.heimes, 2017-09-08 21:06
PR 5395 merged christian.heimes, 2018-01-28 20:20
Messages (7)
msg301731 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-08 21:04
Python should no longer attempt to verify hostname and ip addresses itself. OpenSSL 1.0.2 and newer is able to verify hostname and IP addresses itself. The new APIs are properly hooked into chain validation step. Hostname matching implements RFC 6125. CN matching and partial wildcards can be tuned with additional. The API is documented here: https://www.openssl.org/docs/man1.0.2/crypto/X509_VERIFY_PARAM_set1_host.html . X509_VERIFY_PARAM_set1_host is available since OpenSSL 1.0.2. LibreSSL 2.5.3+ implement the proper bits and pieces, too.


Why should we use OpenSSL rather than matching hostnames ourselves?

In the past, OpenSSL did not contain any code to perform host name
matching. Application were required to role their own implementation.
This caused code duplication and various security issues, because
it is far from trivial to cover all edge cases. Python had multiple
security issues just caused by incorrect or buggy hostname matching:

* Until Python 3.2 and 2.7.9, the ssl module was not capable of
  performing host name matching. ``ssl.match_hostname()`` was
  introduced in 3.2.0 and later back-ported to 2.7.9.
* Issue #12000: Subject CN was ignored when a subject alternative
  name extension (SAN) was present without dNSName entries, thus
  violating RFC 2818.
* CVE-2013-2099: Multiple wildcard characters could be abused
  for Denial-of-Service attack in the re module.
* Issue #17997: RFC 2818 was superseded by RFC 6125, which no longer
  allows multiple wildcard characters. Wildcards are only supported
  in the left-most label.
* Issue #17997: ``ssl.match_hostname()`` did not implement partial
  wildcards of international domain names correctly.
* Issue #18709: The ssl module used an inappropriate OpenSSL function
  to convert host names from ASN.1 to strings. A host name with an
  embedded NULL byte could be abused to trick validation.
* Issue #17305: The ssl module does not handle IDNA 2008-encoded
  host names correctly. It converts from IDN A-label (ASCII
  compatible encoding) to IDN U-label (unicode) with Python's idna
  encoding, which is IDNA 2003-only.
* Issue #30141: The host name is not verified when a SSLSocket is
  created with ``do_handshake_on_connect=False`` and the application
  causes an implicit handshake w/o calling do_handshake() explicitly.
* A SSLSocket performs host name matching *after* the handshake and
  during the handshake. In case of an invalid host name, a client
  is suppose to abort the connection with appropriate TLS alert.
  This causes two problem. For one the server is not informed about
  a problem with the certificate. Also an invalid host name does not
  prevent the client from sending a TLS client authentication
  cert to a malicious server. The cert typically contains personal
  information like username and department.
msg310240 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-18 14:07
Bad news, LibreSSL is the worst. Even the latest release 2.6.4 does not implement https://www.openssl.org/docs/man1.0.2/crypto/X509_VERIFY_PARAM_set1_host.html or X509_VERIFY_PARAM_set_hostflags(). I don't get why it provides X509_check_host() but not X509_VERIFY_PARAM_set1_host().
msg310241 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-18 14:16
https://github.com/libressl-portable/portable/issues/381
msg310862 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-27 14:51
New changeset 61d478c71c5341cdc54e6bfb4ace4252852fd972 by Christian Heimes in branch 'master':
bpo-31399: Let OpenSSL verify hostname and IP address (#3462)
https://github.com/python/cpython/commit/61d478c71c5341cdc54e6bfb4ace4252852fd972
msg311098 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-01-29 11:42
It seems like the commit 61d478c71c5341cdc54e6bfb4ace4252852fd972 introduced a regression in test_ftplib: bpo-32706. Can you please take a look?
msg311129 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-29 13:25
New changeset 66e5742becce38e69a8f09e5f7051445fc57e92e by Christian Heimes in branch 'master':
bpo-28414: ssl module idna test (#5395)
https://github.com/python/cpython/commit/66e5742becce38e69a8f09e5f7051445fc57e92e
msg312893 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-02-26 08:41
The feature has been implemented. I'll take care of the failing tests in #32706.
History
Date User Action Args
2018-02-26 08:41:39christian.heimessetstatus: open -> closed
versions: + Python 3.8
messages: + msg312893

resolution: fixed
stage: patch review -> resolved
2018-02-26 08:22:40christian.heimeslinkissue30141 superseder
2018-01-29 13:30:07Socobsetnosy: + Socob
2018-01-29 13:25:15christian.heimessetmessages: + msg311129
2018-01-29 11:42:29vstinnersetnosy: + vstinner
messages: + msg311098
2018-01-28 20:20:43christian.heimessetpull_requests: + pull_request5231
2018-01-27 14:51:41christian.heimessetmessages: + msg310862
2018-01-18 14:16:40christian.heimessetmessages: + msg310241
2018-01-18 14:07:00christian.heimessetmessages: + msg310240
2017-09-15 16:31:07jwilksetnosy: + jwilk
2017-09-08 21:06:37christian.heimessetkeywords: + patch
pull_requests: + pull_request3454
2017-09-08 21:04:46christian.heimescreate