Title: SSL match_hostname fails for internationalized domain names
Type: enhancement Stage:
Components: asyncio, SSL Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: Socob, abracadaber, christian.heimes, kedare, yselivanov
Priority: normal Keywords:

Created on 2016-10-11 08:02 by abracadaber, last changed 2017-04-21 10:58 by kedare.

Messages (7)
msg278466 - (view) Author: Anton Sychugov (abracadaber) Date: 2016-10-11 08:02
In accordance with
"If the DNS domain name portion of a reference identifier is an internationalized domain name, then an implementation MUST convert any U-labels [IDNA-DEFS] in the domain name to A-labels before checking the domain name."
The question is: Where in python stdlib should it to convert domain name from U-label to A-label? Should it be in ssl._dnsname_match, e.g.:
hostname = hostname.encode('idna').decode('utf-8')
Or should it be at ssl._dnsname_match caller level?

I found that error appears after using ssl.SSLContext.wrap_bio, which in turn uses internal newPySSLSocket, which in turn always decode server_hostname through:
PySSLSocket *self;
PyObject *hostname = PyUnicode_Decode(server_hostname, strlen(server_hostname), "idna", "strict");
self->server_hostname = hostname;
In this way, SSLSocket always contains U-label in its server_hostname field, and ssl._dnsname_match falis with "ssl.CertificateError: hostname ... doesn't match either of ..."

And i don't understand where is a bug, or is it a bug.
msg278483 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-10-11 13:20
Thanks for bringing this to my attention. I can confirm that the code is broken. Further more there are no tests for IDN for server_hostname.

* server_hostname must be an IDN U-label (locälhost)
* SSL handshake correctly converts and sends TLS SNI as IDN A-label (xn--loclhost-2za)
* getpeercert() returns DNS SAN as IDN A-label. It's less than ideal but required.
* the serverhostname_callback is called with IDN U-label
* match_hostname() is called with IDN U-label

The bug is clearly in match_hostname(). The function fails to convert the hostname U-label to A-label before it compares the certificate.

I have a rough draft of a patch here

By the way IDNA support in Python is broken in general, #17305. We still don't support the latest IDNA standard from 2008 (!). IDNA 2003 is not compatible with German, Greek, Farsi and Sinhalese domains,
msg278488 - (view) Author: Anton Sychugov (abracadaber) Date: 2016-10-11 13:42
Yes, I misspelled, match_hostname() fails with ssl.CertificateError.
msg278519 - (view) Author: Anton Sychugov (abracadaber) Date: 2016-10-12 08:07
Christian, thanks a lot for your comment and for patch you provide. It becomes much clearer.
I'll be watching for #17305.
msg279165 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-10-21 21:55
Christian, what's the status on this one?
msg279920 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-11-02 11:02
It's a big, complicated mess. I can't implement IDN support correctly because Python lacks UTS#46 and IDNA2008 support. I just found out that IDNA 2008 is not enough because it does not provide a case mapping. Lack of case mapping broke my fix for curl CVE-2016-8625.

At the moment IDN support is broken in a sane way: it just doesn't work and fails.

A partial fix will introduce security issues. lists "www.sparkasse-gieß" as a critical example. It's the domain of a German savings and loan bank.
msg292025 - (view) Author: Mathieu Poussin (kedare) Date: 2017-04-21 10:58
Hello Christian.

Is there any update about this issue ?
Do we have any alternative to avoid this problem ?

Thank you.
Date User Action Args
2017-04-21 10:58:59kedaresetnosy: + kedare
messages: + msg292025
2017-01-09 18:33:23Socobsetnosy: + Socob
2016-11-02 11:02:28christian.heimessetmessages: + msg279920
2016-10-21 21:55:32yselivanovsetmessages: + msg279165
2016-10-12 08:07:12abracadabersetmessages: + msg278519
2016-10-11 16:49:32gvanrossumsetnosy: - gvanrossum
2016-10-11 13:42:10abracadabersetmessages: + msg278488
2016-10-11 13:20:32christian.heimessetversions: + Python 2.7, Python 3.6, Python 3.7, - Python 3.4
2016-10-11 13:20:13christian.heimessetmessages: + msg278483
2016-10-11 09:05:21abracadabersetassignee: christian.heimes

components: + SSL
nosy: + christian.heimes
2016-10-11 08:23:36abracadabersettype: enhancement
2016-10-11 08:02:36abracadabercreate