Created on 2013-05-17 14:04 by christian.heimes, last changed 2014-02-23 20:43 by jwilk.
|match_hostname_RFC6125.patch||christian.heimes, 2013-08-12 12:47||review|
|msg189454 - (view)||Author: Christian Heimes (christian.heimes) *||Date: 2013-05-17 14:04|
Python's ssl.match_hostname() does sub string matching as specified in RFC 2818: Names may contain the wildcard character * which is considered to match any single domain name component or component fragment. E.g., *.a.com matches foo.a.com but not bar.foo.a.com. f*.com matches foo.com but not bar.com. The RFC doesn't specify how internationalized domain names shoould be handled because it predates RFC 5890 for IDNA by many year. IDNA are prefixed with "xn--", e.g. u"götter.example.de".encode("idna") == "xn--gtter-jua.example.de". This can result into false positive matches for a rule like "x*.example.de". Chrome has special handling for IDN prefix in X509Certificate::VerifyHostname() http://src.chromium.org/viewvc/chrome/trunk/src/net/cert/x509_certificate.cc Also see #17980
|msg189514 - (view)||Author: Antoine Pitrou (pitrou) *||Date: 2013-05-18 14:10|
Actually, I don't this is a bug: match_hostname() expects str data, and therefore IDNA-decoded domain names: >>> b"xn--gtter-jua.example.de".decode("idna") 'götter.example.de' Doing the matching on the decoded domain name should be safe. Then it very much depends on whether the data you've got was IDNA-decoded, or naïvely ASCII-decoded, and I don't think the Python stdlib is very consistent here. Looking at the socket module, gethostbyaddr and getnameinfo seem to use ASCII decoding...
|msg189516 - (view)||Author: Christian Heimes (christian.heimes) *||Date: 2013-05-18 14:35|
It's called "internationalized domain name for APPLICATIONS". ;) It's up to the application to interpret the ASCII text as IDNA encoded FQDNs. As far as I know DNS, SSL's CNAME and OS interfaces etc. always use ASCII labels. It's an elegant solution. Just the UI part of an application needs to understand IDNA. http://tools.ietf.org/html/rfc6125#section-6.4.2 If the DNS domain name portion of a reference identifier is an internationalized domain name, then an implementation MUST convert any U-labels [IDNA-DEFS] in the domain name to A-labels before checking the domain name. In accordance with [IDNA-PROTO], A-labels MUST be compared as case-insensitive ASCII. Each label MUST match in order for the domain names to be considered to match, except as supplemented by the rule about checking of wildcard labels (Section 6.4.3; but see also Section 7.2 regarding wildcards in internationalized domain names). Coincidentally the same RFC contains matching rules for wild card certs http://tools.ietf.org/html/rfc6125#section-6.4.3 If a client matches the reference identifier against a presented identifier whose DNS domain name portion contains the wildcard character '*', the following rules apply: 1. The client SHOULD NOT attempt to match a presented identifier in which the wildcard character comprises a label other than the left-most label (e.g., do not match bar.*.example.net). 2. If the wildcard character is the only character of the left-most label in the presented identifier, the client SHOULD NOT compare against anything but the left-most label of the reference identifier (e.g., *.example.com would match foo.example.com but not bar.foo.example.com or example.com). 3. The client MAY match a presented identifier in which the wildcard character is not the only character of the label (e.g., baz*.example.net and *baz.example.net and b*z.example.net would be taken to match baz1.example.net and foobaz.example.net and buzz.example.net, respectively). However, the client SHOULD NOT attempt to match a presented identifier where the wildcard character is embedded within an A-label or U-label [IDNA-DEFS] of an internationalized domain name [IDNA-PROTO].
|msg189527 - (view)||Author: Antoine Pitrou (pitrou) *||Date: 2013-05-18 16:02|
> It's called "internationalized domain name for APPLICATIONS". ;) It's > up to the application to interpret the ASCII text as IDNA encoded > FQDNs. As far as I know DNS, SSL's CNAME and OS interfaces etc. always > use ASCII labels. It's an elegant solution. Just the UI part of an > application needs to understand IDNA. The socket module already decodes to/encodes from IDNA in places (e.g. gethostname()). We need a consistent policy in the stdlib; I would like Martin's advice on this.
|msg189909 - (view)||Author: Christian Heimes (christian.heimes) *||Date: 2013-05-24 13:09|
I finally found the correct RFC for wildcard matching. I think our implementation violates some recommendations. http://tools.ietf.org/html/rfc6125#section-6.4.2 http://tools.ietf.org/html/rfc6125#section-6.4.3
|msg189928 - (view)||Author: Martin v. Löwis (loewis) *||Date: 2013-05-24 20:23|
As a policy, the standard library should accept non-ASCII host names ("U-labels") wherever possible. I.e the hostname parameter of match_hostname should allow for U-labels (as well as A-labels). When returning names, it should always return the data "as-is", which typically means A-labels. Anybody wanting to display U-labels will need to decode them explicitly. I believe that the matching of IDNA names doesn't currently happen according to 6.4.2 of RFC 6125, however, this is not actually the issue that Christian reported (which was only about wildcard matching). I suggest to create a separate issue for that. As for 6.4.3: I find the text to be quite ill-formulated. Specifically, I'm referring to the sentence However, the client SHOULD NOT attempt to match a presented identifier where the wildcard character is embedded within an A-label or U-label [IDNA-DEFS] of an internationalized domain name [IDNA-PROTO]. First, in the context of X.509, a wildcard *cannot* be embedded "with an ... U-label"; the certificate can only possibly contain A-labels (because the datatype of dNSName is IA5String). Second, as written, it *does* allow to match 'götter.example.de' against "x*.example.de", since "x*.example.de" is not an A-label. An A-label is defined as An "A-label" is the ASCII-Compatible Encoding (ACE, see Section 220.127.116.11) form of an IDNA-valid string. It must be a complete label: IDNA is defined for labels, not for parts of them and not for complete domain names. This means, by definition, that every A-label will begin with the IDNA ACE prefix, "xn--" (see Section 18.104.22.168), followed by a string that is a valid output of the Punycode algorithm [RFC3492] and hence a maximum of 59 ASCII characters in length. The prefix and string together must conform to all requirements for a label that can be stored in the DNS including conformance to the rules for LDH labels (Section 2.3.1). If and only if a string meeting the above requirements can be decoded into a U-label is it an A-label. Since an A-label is required to conform to the LDH label syntax, it cannot possibly contain the asterisk (LDH labels can only contain letters, digits, and the hyphen. Hence, the entire requirement is irrelevant (as literally written). They might mean something else, but I cannot guess what it is that they mean. I disagree with the classification of this issue as critical. It does not involve a crash, a serious regression, or a breakage of a very important API.
|msg194950 - (view)||Author: Christian Heimes (christian.heimes) *||Date: 2013-08-12 12:47|
Ryan Sleevi of the Google Chrome Security Team has informed us about another issue that is caused by our failure to implement RFC 6125 wildcard matching rules. RFC 6125 allows only one wildcard in the left-most fragment of a hostname. For security reasons matching rules like *.*.com should be not supported. For wildcards in internationalized domain names I have followed the piece of advice "In the face of ambiguity, refuse the temptation to guess.". A substring wildcard does no longer match an IDN A-label fragment. '*' still matches a full punycode fragment but 'x*' no longer matches 'xn--foo'. I copied the idea from Chrome's matching code: http://src.chromium.org/viewvc/chrome/trunk/src/net/cert/x509_certificate.cc?revision=212341#l640 // * must not match a substring of an IDN A label; just a whole fragment. if (reference_host.starts_with("xn--") && !(pattern_begin.empty() && pattern_end.empty())) continue; The relevant RFC section for the patch are http://tools.ietf.org/html/rfc6125#section-6.4.3 http://tools.ietf.org/html/rfc2818#section-3.1 http://tools.ietf.org/html/rfc2459#section-22.214.171.124 http://tools.ietf.org/html/rfc5280#section-7
|msg195058 - (view)||Author: Christian Heimes (christian.heimes) *||Date: 2013-08-13 09:10|
Affected versions: - Python 3.2 (< 3.2.5) - Python 3.3 (< 3.3.3) - Python 3.4a1 - requests < 1.2.3 https://pypi.python.org/pypi/requests - backports.ssl_match_hostname (<3.2a3) https://pypi.python.org/pypi/backports.ssl_match_hostname/ - urllib3 < 1.6 https://github.com/shazow/urllib3 - bzr - setuptools - tornado - pip
|msg196823 - (view)||Author: Toshio Kuratomi (a.badger) *||Date: 2013-09-03 03:59|
So, is this a security issue? I've been wondering if I should apply the attached patch to the backports-ssl_match_hostname module on pypi. I was hoping there'd be some information here as to whether this will be going into the stdlib in the future. Thus far, ssl_match_hostname has just been a backport of the match_hostname function but if this is a security problem, I could press for us to diverge from the python3 stdlib. It would be easier to make the case if this is seen as a critical problem that will need to be fixed even if the current patch might not be the eventual fix.
|msg200724 - (view)||Author: Christian Heimes (christian.heimes) *||Date: 2013-10-21 08:54|
Yes, it's a security issue. But the patch would changes the behavior of the function. The current function conforms to RFC 2818. The patch implements RFC 6125, which is more restrictive.
|msg201422 - (view)||Author: Roundup Robot (python-dev)||Date: 2013-10-27 06:38|
New changeset 10d0edadbcdd by Georg Brandl in branch '3.3': Issue #17997: Change behavior of ``ssl.match_hostname()`` to follow RFC 6125, http://hg.python.org/cpython/rev/10d0edadbcdd
|msg201431 - (view)||Author: Georg Brandl (georg.brandl) *||Date: 2013-10-27 06:46|
Also merged to default.
|msg203159 - (view)||Author: Christian Heimes (christian.heimes) *||Date: 2013-11-17 14:08|
Python 3.2 hasn't been fixed yet. Should acquire a CVE for the issue?
|msg207172 - (view)||Author: Martin v. Löwis (loewis) *||Date: 2014-01-02 16:44|
Just to clarify the status of this issue: it *only* blocks 3.2.
|2014-01-02 21:26:49||loewis||set||priority: normal -> release blocker|
|2014-01-02 16:44:07||loewis||set||priority: release blocker -> normal|
messages: + msg207172
|2013-11-17 14:08:01||christian.heimes||set||messages: + msg203159|
versions: - Python 3.3, Python 3.4
messages: + msg201422
|2013-10-21 08:54:40||christian.heimes||set||priority: normal -> release blocker|
nosy: + larry, georg.brandl
messages: + msg200724
messages: + msg196823
|2013-08-13 09:10:05||christian.heimes||set||messages: + msg195058|
versions: + Python 3.2
messages: + msg194950
keywords: + patch
stage: needs patch -> patch review
|2013-05-25 10:50:29||pitrou||set||priority: critical -> normal|
|2013-05-24 20:23:20||loewis||set||messages: + msg189928|
|2013-05-24 13:09:31||christian.heimes||set||messages: + msg189909|
messages: + msg189527
|2013-05-18 14:35:44||christian.heimes||set||messages: + msg189516|
|2013-05-18 14:10:30||pitrou||set||messages: + msg189514|