Unable to establish SSL connections using company's private certificates where their SANs (Subject Alternative Names) contain at least one DNS Name that starts with white spaces.

Attempting to establish SSL connection would result in Exception:
SSLCertVerificationError("partial wildcards in leftmost label are not supported: '   *.x.y.com'.")

This situation made us co-depended on SecOps in a big company where ultimately all other none-python apps weren't effected by that change they made and thus couldn't or wouldn't fix the problem on their side for us. (We were at their mercy!)


I originally encountered this bug @ Python 3.7 and fixed it manually on my own local Python environment.

As the bug seems to be still unfixed to date, I publish this issue.

A small and simple fix will follow shortly on github.

This is not a bug in Python but a misconfiguration on your side. A workaround for a misconfiguration doesn't belong into upstream code. The certificate validation code is security-sensitive and I don't feel comfortable to add unnecessary string transformation to it. The code refuses bad wildcards because we have had more than one CVE related to wildcard matching.

Besides the ssl.match_hostname() function is deprecated and no longer used. Starting with Python 3.7 the ssl module uses OpenSSL to verify host names.

I suggest that you either ship this fix locally with your app. Or talk to IT again and have them replace the wrong certificate with a correct one that does not violate the standards.

Hi,

Judging by your comment, I think there is a an unfortnate misunderstanding.

If you'd be kind enough, please let me explain:

1. The issue I had was indeed on Python 3.7, using the highly used
"requests" library. Also my change was -not- applied on the deprecated
ssl.match_hostname()
but on _dnsname_match() which is a method of another inner class.
My point is that it's still releveant.

2. Although there is a thin line here, it is not a configuration issue by
its classic meaning. It's an outside condition in a production environment
that unpatches Python code cannot handle and thus implicating that Python
is less stable and mistake tolerant than C# and Java (those are the other
more used languages in the company which weren't effected by this problem
-- for some who wouldn't bother like me to patch python source code, this
could be a deal breaker to move to another language).

3. It's a very simple fix that only removes white spaces (empty chars) from
start and end of the DNS before applying all the other tests on it. In fact
by assuring the input of the DNS name, our code becomes -more- secure. In
current state, a missed type DNS name encoded in the certifications could
cause unknowingly Python deamons to stop transmiting data. Also as humans
tend to make such naive errors, a mallicious party could make an attack be
seemless and  be discarded as human error. And if that doesn't convince
you, we can say at the very least the service we provide with our App
becomes unusable and unavailable to clients and for some that could cost
time and money and Python may take the blame as unreliable as compared to
other languages.

4. The thrown exception can be misleading: The exception says that the
problem is a partial wildcard. However the problem is white spaces which
can be difficult to spot. White spaces cannot be part of DNS names thus it
makes no sense to ackonwledge them or refer to them or event test them as
any other legit legal char. Also this is unpredictible to the programmer as
he wouldn't think such a basic trim/strip of white spaces wouldn't happen
in the core of the SSL code what's worse, it can't be handled conventialy
with catching the exception. While a programmer can edit Python source code
to it's needs,  they really shouldn't mess with it for more than a short
term use. Declining the change dooms me for example to always add this
change to projects using SSL.

Thank you for your time.
I truely hope we can resolve this.
D.K

On Wed, Aug 14, 2019, 08:09 Christian Heimes <report@bugs.python.org> wrote:

>
> Christian Heimes <lists@cheimes.de> added the comment:
>
> This is not a bug in Python but a misconfiguration on your side. A
> workaround for a misconfiguration doesn't belong into upstream code. The
> certificate validation code is security-sensitive and I don't feel
> comfortable to add unnecessary string transformation to it. The code
> refuses bad wildcards because we have had more than one CVE related to
> wildcard matching.
>
> Besides the ssl.match_hostname() function is deprecated and no longer
> used. Starting with Python 3.7 the ssl module uses OpenSSL to verify host
> names.
>
> I suggest that you either ship this fix locally with your app. Or talk to
> IT again and have them replace the wrong certificate with a correct one
> that does not violate the standards.
>
> ----------
> resolution:  -> rejected
> stage: patch review -> resolved
> status: open -> closed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue37845>
> _______________________________________
>

On 14/08/2019 15.37, David K. wrote:
> 
> David K. <dikaveman@gmail.com> added the comment:
> 
> Hi,
> 
> Judging by your comment, I think there is a an unfortnate misunderstanding.
> 
> If you'd be kind enough, please let me explain:
> 
> 1. The issue I had was indeed on Python 3.7, using the highly used
> "requests" library. Also my change was -not- applied on the deprecated
> ssl.match_hostname()
> but on _dnsname_match() which is a method of another inner class.
> My point is that it's still releveant.

Except it's not revelant any more because that function is no longer
used by CPython's ssl module. Neither match_hostname nor any of its
helper functions are called by code in the ssl module. Since 3.7 all
hostname verification is now performed by OpenSSL code directly.
ssl.match_hostname will be removed in 3.9 or 3.10.

Latest urllib3 and requests don't use ssl.match_hostname() either. The
urllib3 package has an older copy of the hostname matching algorithm in
urllib3.packages.ssl_match_hostname. It should not be used with modern
Python.

> 2. Although there is a thin line here, it is not a configuration issue by
> its classic meaning. It's an outside condition in a production environment
> that unpatches Python code cannot handle and thus implicating that Python
> is less stable and mistake tolerant than C# and Java (those are the other
> more used languages in the company which weren't effected by this problem
> -- for some who wouldn't bother like me to patch python source code, this
> could be a deal breaker to move to another language).

Your setup has a misconfigured X.509 certificate with a SAN entry that
violates standards for certificates. You are asking me to introduce a
security into Python as a workaround for the misconfiguration.

The algorithm in match_hostname() and _dnsname_match() implements RFC
6125, section 6.4.3, in a strict way. Python not only refuses to match
invalid wildcard entries, it also fails hard on RFC 6125 violations.

> 3. It's a very simple fix that only removes white spaces (empty chars) from
> start and end of the DNS before applying all the other tests on it. In fact
> by assuring the input of the DNS name, our code becomes -more- secure. In
> current state, a missed type DNS name encoded in the certifications could
> cause unknowingly Python deamons to stop transmiting data. Also as humans
> tend to make such naive errors, a mallicious party could make an attack be
> seemless and  be discarded as human error. And if that doesn't convince
> you, we can say at the very least the service we provide with our App
> becomes unusable and unavailable to clients and for some that could cost
> time and money and Python may take the blame as unreliable as compared to
> other languages.

You view the fix as simple and harmless. I see it as a violation of
standards and a security bug. X.509 certs are complex and fragile
beasts. Python have had several CVEs in the hostname matching code
because we didn't implement it correctly. Certificates are also used in
legal contracts, e.g. to legally sign documents or establish mutual
trust. You cannot just modify the content of a certificate.

Since you are worried about the reliability of Python and started
talking about money, have you considered to donate money to the PSF?
Python is maintained by unpaid volunteers. Donations to the Python
Software Foundation help. (Disclaimer: No money in the world will change
my opinion about "dn.strip()".)

> 4. The thrown exception can be misleading: The exception says that the
> problem is a partial wildcard. However the problem is white spaces which
> can be difficult to spot. White spaces cannot be part of DNS names thus it
> makes no sense to ackonwledge them or refer to them or event test them as
> any other legit legal char. Also this is unpredictible to the programmer as
> he wouldn't think such a basic trim/strip of white spaces wouldn't happen
> in the core of the SSL code what's worse, it can't be handled conventialy
> with catching the exception. While a programmer can edit Python source code
> to it's needs,  they really shouldn't mess with it for more than a short
> term use. Declining the change dooms me for example to always add this
> change to projects using SSL.

The problem is not the code but the certificate. If you don't care about
valid certificate, then maybe disable hostname / certificate
verification. Or role your own verification. There is no need to modify
any source code for that.

OK,

I see your point :)

Modification of the original certificiation is legally problematic.

Much thanks for the patience and time to explain,
D.K

On Wed, Aug 14, 2019, 17:23 Christian Heimes <report@bugs.python.org> wrote:

>
> Christian Heimes <lists@cheimes.de> added the comment:
>
> On 14/08/2019 15.37, David K. wrote:
> >
> > David K. <dikaveman@gmail.com> added the comment:
> >
> > Hi,
> >
> > Judging by your comment, I think there is a an unfortnate
> misunderstanding.
> >
> > If you'd be kind enough, please let me explain:
> >
> > 1. The issue I had was indeed on Python 3.7, using the highly used
> > "requests" library. Also my change was -not- applied on the deprecated
> > ssl.match_hostname()
> > but on _dnsname_match() which is a method of another inner class.
> > My point is that it's still releveant.
>
> Except it's not revelant any more because that function is no longer
> used by CPython's ssl module. Neither match_hostname nor any of its
> helper functions are called by code in the ssl module. Since 3.7 all
> hostname verification is now performed by OpenSSL code directly.
> ssl.match_hostname will be removed in 3.9 or 3.10.
>
> Latest urllib3 and requests don't use ssl.match_hostname() either. The
> urllib3 package has an older copy of the hostname matching algorithm in
> urllib3.packages.ssl_match_hostname. It should not be used with modern
> Python.
>
> > 2. Although there is a thin line here, it is not a configuration issue by
> > its classic meaning. It's an outside condition in a production
> environment
> > that unpatches Python code cannot handle and thus implicating that Python
> > is less stable and mistake tolerant than C# and Java (those are the other
> > more used languages in the company which weren't effected by this problem
> > -- for some who wouldn't bother like me to patch python source code, this
> > could be a deal breaker to move to another language).
>
> Your setup has a misconfigured X.509 certificate with a SAN entry that
> violates standards for certificates. You are asking me to introduce a
> security into Python as a workaround for the misconfiguration.
>
> The algorithm in match_hostname() and _dnsname_match() implements RFC
> 6125, section 6.4.3, in a strict way. Python not only refuses to match
> invalid wildcard entries, it also fails hard on RFC 6125 violations.
>
> > 3. It's a very simple fix that only removes white spaces (empty chars)
> from
> > start and end of the DNS before applying all the other tests on it. In
> fact
> > by assuring the input of the DNS name, our code becomes -more- secure. In
> > current state, a missed type DNS name encoded in the certifications could
> > cause unknowingly Python deamons to stop transmiting data. Also as humans
> > tend to make such naive errors, a mallicious party could make an attack
> be
> > seemless and  be discarded as human error. And if that doesn't convince
> > you, we can say at the very least the service we provide with our App
> > becomes unusable and unavailable to clients and for some that could cost
> > time and money and Python may take the blame as unreliable as compared to
> > other languages.
>
> You view the fix as simple and harmless. I see it as a violation of
> standards and a security bug. X.509 certs are complex and fragile
> beasts. Python have had several CVEs in the hostname matching code
> because we didn't implement it correctly. Certificates are also used in
> legal contracts, e.g. to legally sign documents or establish mutual
> trust. You cannot just modify the content of a certificate.
>
> Since you are worried about the reliability of Python and started
> talking about money, have you considered to donate money to the PSF?
> Python is maintained by unpaid volunteers. Donations to the Python
> Software Foundation help. (Disclaimer: No money in the world will change
> my opinion about "dn.strip()".)
>
> > 4. The thrown exception can be misleading: The exception says that the
> > problem is a partial wildcard. However the problem is white spaces which
> > can be difficult to spot. White spaces cannot be part of DNS names thus
> it
> > makes no sense to ackonwledge them or refer to them or event test them as
> > any other legit legal char. Also this is unpredictible to the programmer
> as
> > he wouldn't think such a basic trim/strip of white spaces wouldn't happen
> > in the core of the SSL code what's worse, it can't be handled conventialy
> > with catching the exception. While a programmer can edit Python source
> code
> > to it's needs,  they really shouldn't mess with it for more than a short
> > term use. Declining the change dooms me for example to always add this
> > change to projects using SSL.
>
> The problem is not the code but the certificate. If you don't care about
> valid certificate, then maybe disable hostname / certificate
> verification. Or role your own verification. There is no need to modify
> any source code for that.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue37845>
> _______________________________________
>