classification
Title: Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute
Type: behavior Stage: resolved
Components: SSL Versions: Python 3.7
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, pedja, pervlad
Priority: normal Keywords:

Created on 2019-01-05 12:24 by pervlad, last changed 2020-01-15 21:29 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
cacerts.pem pervlad, 2019-01-07 18:44 pub certs
Messages (11)
msg333054 - (view) Author: Vladimir Perić (pervlad) Date: 2019-01-05 12:24
In Python 3.7.1 on Windows 10 ssl library function call ssl.create_default_context(ssl.Purpose.SERVER_AUTH) raises an ssl error:

File "C:\Python37\lib\ssl.py", line 471, in _load_windows_store_certs
    self.load_verify_locations(cadata=certs)
ssl.SSLError: nested asn1 error (_ssl.c:3926)

In Python 3.6.4 same function call raises no error.
msg333055 - (view) Author: Vladimir Perić (pervlad) Date: 2019-01-05 13:36
Same outcome in Python 3.7.2.
See first comment for detailed explanation of issue.
msg333158 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-01-07 14:49
> self.load_verify_locations(cadata=certs)
> ...
> ssl.SSLError: nested asn1 error (_ssl.c:3926)

It seems like one of your certificate is invalid.

> In Python 3.6.4 same function call raises no error.

We frequently update OpenSSL in Python. You can get OpenSSL version using:

$ python3
Python 3.7.2 (default, Jan  3 2019, 09:14:01) 
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.1.1 FIPS  11 Sep 2018'
>>> ssl.OPENSSL_VERSION_INFO
(1, 1, 1, 0, 15)
>>> ssl.OPENSSL_VERSION_NUMBER
269488143
>>> hex(ssl.OPENSSL_VERSION_NUMBER)
'0x1010100f'
msg333159 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-01-07 14:52
Would it be possible to attach the certification to the issue so someone can try to reproduce the issue? (but don't attach any private key ;-))
msg333168 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-01-07 15:27
The certs are coming from Windows' trust store. Could you please dump the trust store for me and attach the result to the bug tracker. The following script is untested but should work. I don't have access to a Windows machine at the moment.

ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
certs = []
for storename in ("CA", "ROOT"):
    certs.append(storename)
    for cert, encoding, trust in ssl.enum_certificates(storename):
        if encoding == "x509_asn":
            if trust is True or ssl.Purpose.SERVER_AUTH.oid in trust:
                try:
                    ctx.load_verify_locations(cadata=cert)
                except Exception as e:
                    certs.append(str(e))
                certs.append(ssl.DER_cert_to_PEM_cert(cert))

with open('cacerts.pem', 'w') as f:
    f.write('\n'.join(certs))
msg333181 - (view) Author: Vladimir Perić (pervlad) Date: 2019-01-07 18:44
Public Certificate file cert.pem is attached.

Version of ssl lib in pythons on my machine:
Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.1.0j  20 Nov 2018'

Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.2q  20 Nov 2018'
msg333183 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-01-07 19:53
Your Windows cert store contains multiple invalid certificates. The first failing certificate is the custom "MUPCA Root", which looks like a certificate from http://ca.mup.gov.rs/sertifikati.html. The serial number seems to be badly formated or padded. There is nothing we can do about erroneous and bad certificates.

$ openssl x509 -in ca.pem 
unable to load certificate
140613019477824:error:0D0E20DD:asn1 encoding routines:c2i_ibuf:illegal padding:crypto/asn1/a_int.c:187:
140613019477824:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:627:Field=serialNumber, Type=X509_CINF
140613019477824:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:627:Field=cert_info, Type=X509
140613019477824:error:0906700D:PEM routines:PEM_ASN1_read_bio:ASN1 lib:crypto/pem/pem_oth.c:33:

$ openssl asn1parse -in ca.pem  
    0:d=0  hl=4 l=1300 cons: SEQUENCE          
    4:d=1  hl=4 l= 764 cons: SEQUENCE          
    8:d=2  hl=2 l=   3 cons: cont [ 0 ]        
   10:d=3  hl=2 l=   1 prim: INTEGER           :02
   13:d=2  hl=2 l=   4 prim: INTEGER           :BAD INTEGER:[00000066]
   19:d=2  hl=2 l=  13 cons: SEQUENCE          
   21:d=3  hl=2 l=   9 prim: OBJECT            :sha1WithRSAEncryption
   32:d=3  hl=2 l=   0 prim: NULL              
   34:d=2  hl=2 l=  83 cons: SEQUENCE          
   36:d=3  hl=2 l=  19 cons: SET               
   38:d=4  hl=2 l=  17 cons: SEQUENCE          
   40:d=5  hl=2 l=   3 prim: OBJECT            :commonName
   45:d=5  hl=2 l=  10 prim: UTF8STRING        :MUPCA Root
   57:d=3  hl=2 l=  29 cons: SET               
   59:d=4  hl=2 l=  27 cons: SEQUENCE          
   61:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
   66:d=5  hl=2 l=  20 prim: UTF8STRING        :MUP Republike Srbije
   88:d=3  hl=2 l=  16 cons: SET               
   90:d=4  hl=2 l=  14 cons: SEQUENCE          
   92:d=5  hl=2 l=   3 prim: OBJECT            :localityName
   97:d=5  hl=2 l=   7 prim: UTF8STRING        :Beograd
  106:d=3  hl=2 l=  11 cons: SET               
  108:d=4  hl=2 l=   9 cons: SEQUENCE          
  110:d=5  hl=2 l=   3 prim: OBJECT            :countryName
  115:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :RS
  119:d=2  hl=2 l=  30 cons: SEQUENCE          
  121:d=3  hl=2 l=  13 prim: UTCTIME           :100227161918Z
  136:d=3  hl=2 l=  13 prim: UTCTIME           :200227161918Z
  ...

$ wget http://ca.mup.gov.rs/MUPCARoot.crt
$ openssl x509 -in MUPCARoot.crt -inform DER
unable to load certificate
140699773712192:error:0D0E20DD:asn1 encoding routines:c2i_ibuf:illegal padding:crypto/asn1/a_int.c:187:
140699773712192:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:627:Field=serialNumber, Type=X509_CINF
140699773712192:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:627:Field=cert_info, Type=X509
msg333184 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-01-07 20:05
OpenSSL 1.1.0 is more strict than OpenSSL 1.0.2. That's why you don't see the issue with Python 3.6 but with 3.7. The problem is explained in https://mta.openssl.org/pipermail/openssl-dev/2016-February/005100.html

The CA has encoded the integer 102 (0x66) as "02 04 00
00 00 66", which violates the DER standard. The correct encoding is "02 01  66".

>>> from asn1crypto.core import Integer
>>> import binascii
>>> binascii.hexlify(Integer(102).dump())
b'020166'
msg333215 - (view) Author: Vladimir Perić (pervlad) Date: 2019-01-08 09:38
Thank you all for this expeditive help. Sorry for taking your time.
I will remove bad certificates from my machine.
Thanks again.

I will try to close this one.
msg333224 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-01-08 11:45
I also checked how other implementations deal with invalid DER encoding. NSS 3.41, Firefox, and Chromium accept the certifiate.

NSS shows the serial number as "102 (0x66)"
Firefox and Chromium display the serial number as "00:00:00:66".

$ echo "password" > passwd
$ certutil -d . -f passwd -N
$ certutil -d . -f passwd -A -n ca -i ../ca.pem -t C,C,C
$ certutil -d . -L -n ca
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 102 (0x66)
        Signature Algorithm: PKCS #1 SHA-1 With RSA Encryption
        Issuer: "C=RS,L=Beograd,O=MUP Republike Srbije,CN=MUPCA Root"
        Validity:
            Not Before: Sat Feb 27 16:19:18 2010
            Not After : Thu Feb 27 16:19:18 2020
        Subject: "C=Re...,L=Beograd,O=MUP Republike Srbije,CN=MUPCA Resursi"
        Subject Public Key Info:
            Public Key Algorithm: PKCS #1 RSA Encryption
            RSA Public Key:
                Modulus:
                    ea:69:46:bc:c7:70:00:d5:f5:32:8d:c7:4e:ad:3a:a5:
                    d3:29:7e:a2:46:12:a9:dd:57:75:b1:49:95:80:20:ed:
                    9b:68:6b:e3:c5:55:d8:64:15:68:42:ab:a3:f7:c0:96:
                    37:08:51:cb:05:ca:b5:99:f6:07:a6:8b:f2:cd:d2:f5:
                    d6:16:12:da:bf:a8:0b:9c:45:5d:ac:79:1d:a8:67:47:
                    ee:7f:83:40:f8:58:00:d5:dd:c4:c9:52:1b:d2:f4:ce:
                    e1:fa:8a:66:d3:18:86:1e:ea:fc:0a:8b:b5:ec:49:cd:
                    86:bf:8b:7e:b0:61:81:ec:ea:99:4f:64:82:96:93:9d:
                    ab:80:7d:a7:27:65:00:d4:12:26:98:45:64:7e:76:0b:
                    98:ff:16:50:49:0c:45:20:82:ce:2e:23:a2:65:3a:b7:
                    44:cd:51:00:d9:bf:e3:1f:de:23:1d:57:e9:32:c3:55:
                    f0:24:af:d4:cf:cd:9e:77:1f:19:7e:1c:03:5b:7a:e4:
                    75:84:3b:d4:1d:e9:23:d6:8c:f2:8f:b2:0d:e3:79:df:
                    9e:03:1e:0e:15:5b:7b:0c:dd:6e:4d:82:86:5a:63:79:
                    64:b5:07:79:dd:fd:08:e3:d6:cb:60:01:fd:82:11:59:
                    2c:8d:22:f8:f9:91:59:b1:cd:12:7b:39:6d:08:82:5d
                Exponent: 65537 (0x10001)
        Signed Extensions:
            Name: Certificate Basic Constraints
            Critical: True
            Data: Is a CA with no maximum path length.

            Name: Certificate Key Usage
            Critical: True
            Usages: Certificate Signing
                    CRL Signing

            Name: Authority Information Access
            Method: PKIX CA issuers access method
            Location: 
                URI: "http://ca.mup.gov.rs/MUPCARoot.crt"

            Name: Certificate Subject Key ID
            Data:
                cb:f9:00:a9:b7:b6:c1:6f:44:43:d0:22:ad:fc:0e:6e:
                cc:8f:f6:0f

            Name: Certificate Authority Key Identifier
            Key ID:
                3f:66:b0:0f:66:fb:f0:10:2e:61:a4:6f:ef:2c:95:8a:
                14:72:6f:71

            Name: CRL Distribution Points
            Distribution point:
                URI: "http://ca.mup.gov.rs/MUPCARoot.crl"

    Signature Algorithm: PKCS #1 SHA-1 With RSA Encryption
msg360050 - (view) Author: Pedja (pedja) Date: 2020-01-15 12:58
This is still an issue. Serious one. People are unable to just remove this certificate as it is needed for everyday use.

It is reasonable that application does not deal with invalid certificates. 
I can understand application to breaks if one tries to use invalid certificate. 

But this is not that case. Application breaks on just enumerating certificates even if it does not need it at all.

It is advisable for application just to skip and ignore invalid certificate unless it is required for application to work.

Please reconsider action on this issue to prevent application breaking when it is not necessary.
History
Date User Action Args
2020-01-15 21:29:37vstinnersetnosy: - vstinner
2020-01-15 12:58:40pedjasetnosy: + pedja
messages: + msg360050
2019-01-08 11:45:02christian.heimessetmessages: + msg333224
2019-01-08 09:38:24pervladsetstatus: open -> closed

messages: + msg333215
stage: resolved
2019-01-07 20:05:02christian.heimessetmessages: + msg333184
2019-01-07 19:53:42christian.heimessetmessages: + msg333183
2019-01-07 18:44:27pervladsetfiles: + cacerts.pem

messages: + msg333181
2019-01-07 15:27:38christian.heimessetmessages: + msg333168
2019-01-07 14:52:14vstinnersetmessages: + msg333159
2019-01-07 14:49:04vstinnersetnosy: + vstinner
messages: + msg333158
2019-01-05 13:36:32pervladsetmessages: + msg333055
2019-01-05 12:24:45pervladcreate