classification
Title: SSL BIO is broken for internationalized domains
Type: behavior Stage: resolved
Components: SSL Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: SSL match_hostname fails for internationalized domain names
View: 28414
Assigned To: christian.heimes Nosy List: asvetlov, christian.heimes, njs, pitrou
Priority: normal Keywords:

Created on 2017-10-26 10:09 by asvetlov, last changed 2017-11-02 08:50 by asvetlov. This issue is now closed.

Messages (3)
msg305042 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-10-26 10:09
`SSLContext.wrap_bio` creates a new `SSLObject` instance with passed `server_hostname`.
The name becomes IDNA-decoded: `'xn--2qq421aovb6v1e3pu.xn--j6w193g'` is converted to `'雜草工作室.香港'` by `SSLObject` constructor.

Than on SSL handshake `ssl.match_hostname()` is called with `sslobject.server_hostname` parameter (`'雜草工作室.香港'` in my example).

But certificate for the site is contains IDNA-encoded DNS names:
```
{'OCSP': ('http://ocsp.comodoca4.com',),
 'caIssuers': ('http://crt.comodoca4.com/COMODOECCDomainValidationSecureServerCA2.crt',),
 'crlDistributionPoints': ('http://crl.comodoca4.com/COMODOECCDomainValidationSecureServerCA2.crl',),
 'issuer': ((('countryName', 'GB'),),
            (('stateOrProvinceName', 'Greater Manchester'),),
            (('localityName', 'Salford'),),
            (('organizationName', 'COMODO CA Limited'),),
            (('commonName',
              'COMODO ECC Domain Validation Secure Server CA 2'),)),
 'notAfter': 'Mar 28 23:59:59 2018 GMT',
 'notBefore': 'Sep 19 00:00:00 2017 GMT',
 'serialNumber': 'FBFE0BF7CACA6DDC15968410BAA1908D',
 'subject': ((('organizationalUnitName', 'Domain Control Validated'),),
             (('organizationalUnitName', 'PositiveSSL Multi-Domain'),),
             (('commonName', 'sni38752.cloudflaressl.com'),)),
 'subjectAltName': (('DNS', 'sni38752.cloudflaressl.com'),
                    ('DNS', '*.1km.hk'),
                    ('DNS', '*.acg-cafe.com'),
                    ('DNS', '*.acgapp.moe'),
                    ('DNS', '*.acgapp.net'),
                    ('DNS', '*.cosmatch.org'),
                    ('DNS', '*.dirimusik.com'),
                    ('DNS', '*.dirimusik.info'),
                    ('DNS', '*.downloadlagi.club'),
                    ('DNS', '*.downloadlaguaz.info'),
                    ('DNS', '*.farmprecision.com'),
                    ('DNS', '*.glowecommercialphotography.co.uk'),
                    ('DNS', '*.hypertechglobal.com'),
                    ('DNS', '*.hypertechglobal.hk'),
                    ('DNS', '*.infoku.download'),
                    ('DNS', '*.inimp3.com'),
                    ('DNS', '*.luciafitness.com.au'),
                    ('DNS', '*.merdeka.news'),
                    ('DNS', '*.promisecos.com'),
                    ('DNS', '*.promisecos.hk'),
                    ('DNS', '*.ps9architects.com'),
                    ('DNS', '*.rubaxeu.gq'),
                    ('DNS', '*.ruth-fox.com'),
                    ('DNS', '*.simmit.net.au'),
                    ('DNS', '*.startss.today'),
                    ('DNS', '*.xn--2qq421aovb6v1e3pu.xn--j6w193g'),
                    ('DNS', '*.xn--hhrw16aw6jizf.xn--j6w193g'),
                    ('DNS', '1km.hk'),
                    ('DNS', 'acg-cafe.com'),
                    ('DNS', 'acgapp.moe'),
                    ('DNS', 'acgapp.net'),
                    ('DNS', 'cosmatch.org'),
                    ('DNS', 'dirimusik.com'),
                    ('DNS', 'dirimusik.info'),
                    ('DNS', 'downloadlagi.club'),
                    ('DNS', 'downloadlaguaz.info'),
                    ('DNS', 'farmprecision.com'),
                    ('DNS', 'glowecommercialphotography.co.uk'),
                    ('DNS', 'hypertechglobal.com'),
                    ('DNS', 'hypertechglobal.hk'),
                    ('DNS', 'infoku.download'),
                    ('DNS', 'inimp3.com'),
                    ('DNS', 'luciafitness.com.au'),
                    ('DNS', 'merdeka.news'),
                    ('DNS', 'promisecos.com'),
                    ('DNS', 'promisecos.hk'),
                    ('DNS', 'ps9architects.com'),
                    ('DNS', 'rubaxeu.gq'),
                    ('DNS', 'ruth-fox.com'),
                    ('DNS', 'simmit.net.au'),
                    ('DNS', 'startss.today'),
                    ('DNS', 'xn--2qq421aovb6v1e3pu.xn--j6w193g'),
                    ('DNS', 'xn--hhrw16aw6jizf.xn--j6w193g')),
 'version': 3}
```

Match `'雜草工作室.香港'` to `('DNS', 'xn--2qq421aovb6v1e3pu.xn--j6w193g')` obviously fails.

I see two possible solutions:

1. Always do IDNA encoding for `server_hostname` stored in ssl object.
2. Do two checks for both IDNA and original server hostname values. I don't sure if certificates always use IDNA-encoded DNS names only.

The fix is trivial, I'll make a Pull Request after choosing what option we want to support. Personally I'm inclined to second one.

P.S.
`requests` library is not affected because it uses `ssl.wrap_socket`.
The bug is reproducible for `asyncio` only (and maybe Tornado with `asyncio` `IOLoop`).
msg305404 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2017-11-02 02:11
I believe https://github.com/python/cpython/pull/3010 is the fix you're looking for.
msg305407 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2017-11-02 08:50
Duplicate of #28414

Nathaniel thanks for Pull Request!
History
Date User Action Args
2017-11-02 08:50:30asvetlovsetstatus: open -> closed
superseder: SSL match_hostname fails for internationalized domain names
messages: + msg305407

resolution: duplicate
stage: resolved
2017-11-02 02:11:08njssetnosy: + njs
messages: + msg305404
2017-10-27 11:47:08asvetlovsetversions: - Python 3.5
2017-10-26 10:25:41asvetlovsettype: behavior
2017-10-26 10:09:19asvetlovcreate