http://buildbot.python.org/all/builders/x86%20Gentoo%20Installed%20with%20X%203.x/builds/593/steps/test/logs/stdio


======================================================================
ERROR: test_logincapa_with_client_certfile (test.test_imaplib.RemoteIMAP_SSLTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/test_imaplib.py", line 972, in test_logincapa_with_client_certfile
    certfile=CERTFILE)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/imaplib.py", line 1280, in __init__
    IMAP4.__init__(self, host, port)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/imaplib.py", line 197, in __init__
    self.open(host, port)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/imaplib.py", line 1293, in open
    IMAP4.open(self, host, port)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/imaplib.py", line 294, in open
    self.sock = self._create_socket()
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/imaplib.py", line 1285, in _create_socket
    server_hostname=self.host)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/ssl.py", line 808, in __init__
    self.do_handshake()
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:749)

RemoteIMAP_SSLTest uses cyrus.andrew.cmu.edu server with the port 993.

Would it be possible to setup an IMAP server on pythontest.net instead of using a server of an university? http://www.cmu.edu/

https://travis-ci.org/python/cpython/jobs/226357867

Oh, the test started to fail on Travis CI as well:

======================================================================

ERROR: test_logincapa_with_client_certfile (test.test_imaplib.RemoteIMAP_SSLTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/build/python/cpython/Lib/test/test_imaplib.py", line 973, in test_logincapa_with_client_certfile

    certfile=CERTFILE)

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 1280, in __init__

    IMAP4.__init__(self, host, port)

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 197, in __init__

    self.open(host, port)

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 1293, in open

    IMAP4.open(self, host, port)

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 294, in open

    self.sock = self._create_socket()

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 1285, in _create_socket

    server_hostname=self.host)

  File "/home/travis/build/python/cpython/Lib/ssl.py", line 401, in wrap_socket

    _context=self, _session=session)

  File "/home/travis/build/python/cpython/Lib/ssl.py", line 808, in __init__

    self.do_handshake()

  File "/home/travis/build/python/cpython/Lib/ssl.py", line 1061, in do_handshake

    self._sslobj.do_handshake()

  File "/home/travis/build/python/cpython/Lib/ssl.py", line 683, in do_handshake

    self._sslobj.do_handshake()

ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:749)

======================================================================

ERROR: test_logincapa_with_client_ssl_context (test.test_imaplib.RemoteIMAP_SSLTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/build/python/cpython/Lib/test/test_imaplib.py", line 979, in test_logincapa_with_client_ssl_context

    self.host, self.port, ssl_context=self.create_ssl_context())

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 1280, in __init__

    IMAP4.__init__(self, host, port)

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 197, in __init__

    self.open(host, port)

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 1293, in open

    IMAP4.open(self, host, port)

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 294, in open

    self.sock = self._create_socket()

  File "/home/travis/build/python/cpython/Lib/imaplib.py", line 1285, in _create_socket

    server_hostname=self.host)

  File "/home/travis/build/python/cpython/Lib/ssl.py", line 401, in wrap_socket

    _context=self, _session=session)

  File "/home/travis/build/python/cpython/Lib/ssl.py", line 808, in __init__

    self.do_handshake()

  File "/home/travis/build/python/cpython/Lib/ssl.py", line 1061, in do_handshake

    self._sslobj.do_handshake()

  File "/home/travis/build/python/cpython/Lib/ssl.py", line 683, in do_handshake

    self._sslobj.do_handshake()

ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:749)

haypo@selma$ openssl s_client -connect cyrus.andrew.cmu.edu:993
CONNECTED(00000003)
depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify return:1
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Organization Validation Secure Server CA
verify return:1
depth=0 C = US, postalCode = 15213, ST = PA, L = Pittsburgh, street = 5000 Forbes, O = Carnegie Mellon University, OU = Carnegie Mellon University, OU = Multi-Domain SSL, CN = cyrus.andrew.cmu.edu
verify return:1
---
Certificate chain
 0 s:/C=US/postalCode=15213/ST=PA/L=Pittsburgh/street=5000 Forbes/O=Carnegie Mellon University/OU=Carnegie Mellon University/OU=Multi-Domain SSL/CN=cyrus.andrew.cmu.edu
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Organization Validation Secure Server CA
 1 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Organization Validation Secure Server CA
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
 2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
 3 s:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF2zCCBMOgAwIBAgIQTkDJp5DZKOpt6Vy2UaZ24DANBgkqhkiG9w0BAQsFADCB
ljELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
A1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxPDA6BgNV
BAMTM0NPTU9ETyBSU0EgT3JnYW5pemF0aW9uIFZhbGlkYXRpb24gU2VjdXJlIFNl
cnZlciBDQTAeFw0xNzA0MjQwMDAwMDBaFw0yMDA0MjMyMzU5NTlaMIHZMQswCQYD
VQQGEwJVUzEOMAwGA1UEERMFMTUyMTMxCzAJBgNVBAgTAlBBMRMwEQYDVQQHEwpQ
aXR0c2J1cmdoMRQwEgYDVQQJEws1MDAwIEZvcmJlczEjMCEGA1UEChMaQ2FybmVn
aWUgTWVsbG9uIFVuaXZlcnNpdHkxIzAhBgNVBAsTGkNhcm5lZ2llIE1lbGxvbiBV
bml2ZXJzaXR5MRkwFwYDVQQLExBNdWx0aS1Eb21haW4gU1NMMR0wGwYDVQQDExRj
eXJ1cy5hbmRyZXcuY211LmVkdTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBAM0VgQbuCBrfWJ0B6YB+TrCD2KDSeOmRyKJkFB+T0fKK658pBNlAxS8i98QC
7tOApMdGXw3OJaiCIoUq6QMyXC1SnnEITi5m1qORLlwu3+FG6548FHonAyMydU/i
JBHs4goA7Cy3QUxMPe+ZZTkWr+lQpElukYqmDao/x4ZERz8mZPI/zPzlNALv3TF1
lcpQUHuI5Kw2G0hymANJY3W0hU8JwOBRAjO4em+XVNKXJD4seYGHp+7YZP40Xua/
YrRSwokUqVy/Y3+ytLH6Amnp3zJKpgFM/XSGy3KcuQCwEa86H/DVj/dN710KOpW5
S1Szza4b+gaCb+BKi2u9bFYB0PcCAwEAAaOCAd4wggHaMB8GA1UdIwQYMBaAFJrz
K9rPrU+2L7sqSEgqErcbQsEkMB0GA1UdDgQWBBR/3PPQ2+NzaAVGAghMELRikx4Q
hTAOBgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggrBgEF
BQcDAQYIKwYBBQUHAwIwUAYDVR0gBEkwRzA7BgwrBgEEAbIxAQIBAwQwKzApBggr
BgEFBQcCARYdaHR0cHM6Ly9zZWN1cmUuY29tb2RvLmNvbS9DUFMwCAYGZ4EMAQIC
MFoGA1UdHwRTMFEwT6BNoEuGSWh0dHA6Ly9jcmwuY29tb2RvY2EuY29tL0NPTU9E
T1JTQU9yZ2FuaXphdGlvblZhbGlkYXRpb25TZWN1cmVTZXJ2ZXJDQS5jcmwwgYsG
CCsGAQUFBwEBBH8wfTBVBggrBgEFBQcwAoZJaHR0cDovL2NydC5jb21vZG9jYS5j
b20vQ09NT0RPUlNBT3JnYW5pemF0aW9uVmFsaWRhdGlvblNlY3VyZVNlcnZlckNB
LmNydDAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuY29tb2RvY2EuY29tMB8GA1Ud
EQQYMBaCFGN5cnVzLmFuZHJldy5jbXUuZWR1MA0GCSqGSIb3DQEBCwUAA4IBAQCf
1U2esEfk12MXwrYYvWI0QJSnMvYG9qu8nVDHwoSGsivr4GoBf5cFhtF6gGGERD2x
8kvCuXbkCtf3nWNffUhs7OXzMapA8eOJr7VtmwQraYaDpFCTZpj2X8rpdyhjIBCb
V1VXdxS56TKiQKN72JmdB+e06+w49ym0WMzBjPxZGncogNA0mvKaquiFWe0j0ZT9
rqfuIIon2P3cg+jZctytejVoMFOLkTxSslxjNDNjDvJ4zQnuH6zuhP/1vkpNGSp3
+J8xSw1ubLU7pN/Kv8tR9KsJcRafFC0iryipaah2zib27zaRgO25eq5/dKqRUa96
77xcSeTyQ/2hwVFLrJjW
-----END CERTIFICATE-----
subject=/C=US/postalCode=15213/ST=PA/L=Pittsburgh/street=5000 Forbes/O=Carnegie Mellon University/OU=Carnegie Mellon University/OU=Multi-Domain SSL/CN=cyrus.andrew.cmu.edu
issuer=/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Organization Validation Secure Server CA
---
Acceptable client certificate CA names
/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Organization Validation Secure Server CA
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 6657 bytes and written 339 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: E02AC176F356733EF7E923E01CC744E42E0FA82633E3CEC3E605E89059AFC469
    Session-ID-ctx: 
    Master-Key: 8B2B2E8287A4CC571B77628A14D2E21A284FEFB57843F6C057BE9BE57CC663C7DFCF6DA7BC260CE41785EE00A45F3310
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket lifetime hint: 86400 (seconds)
    TLS session ticket:
    0000 - dd 0a e4 c7 07 68 8e de-90 9d 90 9c 1c cb 18 8a   .....h..........
    0010 - da 1c eb 77 9a f5 0b 0c-28 1a 43 4e cc 4d fa 1a   ...w....(.CN.M..
    0020 - 9e 3e 50 7d d5 5c 99 7f-d8 c7 2a 80 8d 3b 88 49   .>P}.\....*..;.I
    0030 - 84 36 33 46 04 be 4c c8-ac 5d 2a 38 d6 4f c7 da   .63F..L..]*8.O..
    0040 - b5 28 05 b2 36 f8 fd 6f-40 0b 70 ea e1 8c e0 59   .(..6..o@.p....Y
    0050 - 58 12 d3 b1 75 7c 99 bc-d7 6c f9 95 ea 33 d5 d9   X...u|...l...3..
    0060 - 41 99 ac 6f 53 75 96 f5-32 52 ac 6f 26 76 c7 28   A..oSu..2R.o&v.(
    0070 - 63 9e 5c 19 c4 fc cd 0c-9e ea 6c d0 9c 50 77 7a   c.\.......l..Pwz
    0080 - ff 28 a4 a1 96 56 1f 45-3c d8 fc 6e 9e 5f 0b 9d   .(...V.E<..n._..
    0090 - 2e 12 42 db f4 56 55 20-ef 87 d4 a7 2d 8f 57 74   ..B..VU ....-.Wt

    Start Time: 1493297318
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE MUPDATE=mupdate://mupdate.andrew.cmu.edu/ AUTH=GSSAPI AUTH=PLAIN AUTH=ANONYMOUS SASL-IR] cyrus.andrew.cmu.edu Cyrus IMAP 3.0.0-beta1-dev3191 server ready

"tlsv1 alert unknown ca" means our client certificate isn't accepted by the server.  I suggest simply skipping the test, unless we find a remote IMAP server that accepts random TLS certs.

https://serverfault.com/questions/793260/what-does-tlsv1-alert-unknown-ca-mean

> "tlsv1 alert unknown ca" means our client certificate isn't accepted by the server.  I suggest simply skipping the test, unless we find a remote IMAP server that accepts random TLS certs.

Wait, the test tries to connect to a public IMAP server using a x509 client certificate? Wow, I'm surprised that it works sometimes :-)

In RemoteIMAP_SSLTest, I see 3 unit tests which use create_ssl_context(), the method which loads the client certificate, but I only see 2 failures, not 3. Should I understand that sometimes the TLS connection succeeded, but sometimes it fails, on the same server? Strange :-)

test_logincapa_with_client_certfile and test_logincapa_with_client_ssl_context both fail accordingly.

test_ssl_context_certfile_exclusive and test_ssl_context_keyfile_exclusive raise another error (ValueError) and succeed expectedly.

>  I suggest simply skipping the test, unless we find a remote IMAP server that accepts random TLS certs.

I tried "imap.gmail.com" server: first, it fails with:

imaplib.IMAP4.error: b'[AUTHENTICATIONFAILED] Invalid credentials (Failure)'

And then:

imaplib.IMAP4.error: b'[ALERT] Web login required: https://support.google.com/mail/answer/78754 (Failure)'

Well, I don't see why an IMAP server would accept our client cert?

Lib/test/keycert3.pem was added by the commit 58ddc9d743d09ee93d5cf46a4de62eab30dad79d (made by Antoine Pitrou): the certificated was *generated* randomly by Lib/test/make_ssl_certs.py (also added by the same commit).

RemoteIMAP_SSLTest was added by commit b1436f185dac249f0f6dc342b2a543ae2d1db63d (Antoine Pitrou) in 2010 with the server: cyrus.andrew.cmu.edu.

So since 2010 until a few days ago, cyrus.andrew.cmu.edu accepted random client certificate, but now the server started to fail... as expected?

I proposed a quick fix to repair our development process (CI) and buildbots, skip the two failing files:
https://github.com/python/cpython/pull/1320/

I would give our more time to decide what to do:

* contact cyrus.andrew.cmu.edu sysadmin to ask him/her to accept our client cert or revert their IMAP config :-)
* setup a new IMAP server accepts this client cert on pythontest.net
* mock a IMAP server in test_imaplib: mock everything (no network), or "real" TCP server doing the TLS handshake? Mock everything doens't make sense for a unit test designed to use "remote" resources?

Setting up an imap server on pythontest is the "right" answer.  Somebody has to do the work, though :)

New changeset 5bccca58b9b2b3a925b16750bedbd907695ea8d7 by Victor Stinner in branch 'master':
bpo-30175: Skip client cert tests of test_imaplib (#1320)
https://github.com/python/cpython/commit/5bccca58b9b2b3a925b16750bedbd907695ea8d7

New changeset 4dc3b9cf2a868a8378ecbc2221a15e7b59e8e944 by Victor Stinner in branch '3.6':
bpo-30175: Skip client cert tests of test_imaplib (#1320) (#1323)
https://github.com/python/cpython/commit/4dc3b9cf2a868a8378ecbc2221a15e7b59e8e944

New changeset 0eda2d43a7d5b262c979b944592999015a822395 by Victor Stinner in branch '3.5':
bpo-30175: Skip client cert tests of test_imaplib (#1320) (#1324)
https://github.com/python/cpython/commit/0eda2d43a7d5b262c979b944592999015a822395

Ok, the test is now skipped on 3.5, 3.6 and master to fix CI and buildbots.

R. David Murray: "Setting up an imap server on pythontest is the "right" answer.  Somebody has to do the work, though :)"

Who work on pytohntest? Where should I ask that?

I think Benjamin is the only one who has done any work on it so far.  I think you'd ask the infrastructure team for access if you want to do so.

Buildbots are now fixed, I close the issue.

I created the issue #30231 to propose to run an IMAP server on pythontest.net, or to remove the two unit tests which are currently skipped.

Thanks for your help Antoine Pitrou!