On my laptop (running OSX 10.8, but I have noticed the same on earlier OSX releases) test_ssl fails:


======================================================================
FAIL: test_connect (test.test_ssl.NetworkedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ronald/Projects/python/rw/default/Lib/test/test_ssl.py", line 650, in test_connect
    s.connect, ("svn.python.org", 443))
AssertionError: SSLError not raised by connect

======================================================================
FAIL: test_connect_with_context (test.test_ssl.NetworkedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ronald/Projects/python/rw/default/Lib/test/test_ssl.py", line 743, in test_connect_with_context
    s.connect, ("svn.python.org", 443))
AssertionError: SSLError not raised by connect

======================================================================
FAIL: test_get_server_certificate (test.test_ssl.NetworkedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ronald/Projects/python/rw/default/Lib/test/test_ssl.py", line 848, in test_get_server_certificate
    _test_get_server_certificate('svn.python.org', 443, SVN_PYTHON_ORG_ROOT_CERT)
  File "/Users/ronald/Projects/python/rw/default/Lib/test/test_ssl.py", line 840, in _test_get_server_certificate
    self.fail("Got server certificate %s for %s:%s!" % (pem, host, port))
AssertionError: Got server certificate -----BEGIN CERTIFICATE-----
MIIEmTCCAoGgAwIBAgIDCyOkMA0GCSqGSIb3DQEBBQUAMHkxEDAOBgNVBAoTB1Jv
b3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZ
Q0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJARYSc3VwcG9y
dEBjYWNlcnQub3JnMB4XDTExMTIyNTIxMzIxNVoXDTEzMTIyNDIxMzIxNVowGTEX
MBUGA1UEAxMOc3ZuLnB5dGhvbi5vcmcwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQDM6un3wTW9+HVJ7KC+/GwL0KAxehug0tw2YoSSX+TGxLyr9AUtBHQk
hCNWhRLewa0WMOY6hxIIQY1Hp6vreDiCbBehjVkEAydlKBzaAsgYCEbCC/ZaMzhv
aaFAiLVeaxAKJsBGUJNz5hGgzd67A6SGz+XK7qDWig4NR5eFrsr3DvjyEM7txMiG
gftGWLkadOuqUQsI20AykBGi+RxmrQIwqO2svGmje89DsWVILdP37PssM2zqRonh
4fUKooei3L43tXbTdHayXc9NtFS7q8T4eUlyWaD+BtP80QQOQFFvi+qZpme9bmYI
7YPX+e86lZtxAKM9nWrP93qc+2nS0MsHAgMBAAGjgYkwgYYwDAYDVR0TAQH/BAIw
ADA0BgNVHSUELTArBggrBgEFBQcDAgYIKwYBBQUHAwEGCWCGSAGG+EIEAQYKKwYB
BAGCNwoDAzALBgNVHQ8EBAMCBaAwMwYIKwYBBQUHAQEEJzAlMCMGCCsGAQUFBzAB
hhdodHRwOi8vb2NzcC5jYWNlcnQub3JnLzANBgkqhkiG9w0BAQUFAAOCAgEAEsw6
TMEXRn8ex1feaSNB5r2/aCLKE33ssGTp5HAyFo04YF2WaGK8HgjEg/fqrGUaSORw
P1ikBNTAUiaKbRQtcIvQPg8Zk4lSiQZ7m6ePs4hKdFVKeOpykwQyTsm+NaGk4xtM
Zk0XsWkzFntG676MT2FGvDCn0XhtdpAGJy59FXQ1mto9j99AvXpcl9Ja7tRYI/mI
goq3Gu6lwIoYSnslOtIWgsj9Wog1Kug7zoGscxY6z2lk15wbQlQbXbdAbVW7RZry
07eGwROLpE8tydRvvCbJp3P7FE+aAk1D3fQi09G0dUJWRmvBoFOVZCfHmGx23Mpz
YTVgGP1bydjjzzhNAqaKctLf6g3UuUwsOJ1pso4Cr/00EDWG1JsRb4wtlpAEPSnb
vjr/TqHMfefeQf4DmgWo1DutKhUBur5t+VukVCN1z44VtKeu8zaNaA3vR2Q+jArl
YxIFGUTuBcDHBx5Zl0Wcauoe3IbIBckBk2+WOBXVAdj00a46XJ4OHesft63HVOLn
vsKqAhwHPgsH0iTrgbO6ky0Xk573aJWz5wrlYin2FzHXsfJkA7U0wesrG4AFdeAm
zICOAFz9bOhyi6t5c0whF6L0W6Gt4cW+tERG+CD++2svdh8JP9QgozoSKd9N33bg
q5qGifkRs3lTJQGn2wDkgepX1vtshWgRXR9QTRw=
-----END CERTIFICATE-----
 for svn.python.org:443!


This machine has the cacert root certificate as a trusted key in the system keychain.

This is using Apple's copy of OpenSSL, I haven't tested yet if the test failure is also present with the upstream edition of OpenSSL.

The failure seems to occur because the test assumes that the OpenSSL library either won't load a CA list at all when ca_cert is not specified, or that the default CA list doesn't contain the cacert.org one.

I can avoid the test failures by always specifying ca_certs in these tests, but point the argument to an empty file. That's probably not the right solution though, hence I haven't included this as a patch.

> The failure seems to occur because the test assumes that the OpenSSL
> library either won't load a CA list at all when ca_cert is not
> specified, or that the default CA list doesn't contain the cacert.org one.

Well, OpenSSL should not implicitly load a CA list when not asked to.
If it does, it means Apple hacked their OpenSSL copy.
(I am further surprised that Apple includes cacert by default in the trusted certificates, while mainstream browsers like Firefox don't)

As Ronald is aware, there is also the issue that Apple has deprecated use of OpenSSL in OS X:

"Although OpenSSL is commonly used in the open source community, OpenSSL does not provide a stable API from version to version. For this reason, although OS X provides OpenSSL libraries, the OpenSSL libraries in OS X are deprecated, and OpenSSL has never been provided as part of iOS. Use of the OS X OpenSSL libraries by applications is strongly discouraged.

If your application depends on OpenSSL, you should compile OpenSSL yourself and statically link a known version of OpenSSL into your application."

In OS X 10.7, for instance, OpenSSL is at 0.9.8r. I think the same is true for 10.8. We should probably bite the bullet here and do as Apple urges, that is, supply our own libssl 1.0.x for the python.org OS X installer builds.

http://developer.apple.com/library/mac/#documentation/security/Conceptual/cryptoservices/GeneralPurposeCrypto/GeneralPurposeCrypto.html

What's rather annoying is that I cannot find OpenSSL on opensource.apple.com, which means we cannot check if they use patches add functionality that our users would like to have. 

One such feature is likely keychain integration (that is, use the CA roots from the user and system keychain instead of a CA root store in the file system). I'm not 100% sure that this functionality is actually present, but as _ssl automaticly finds a CA root certificate that I have added to the system keychain gives a pretty clear indication.

BTW. It might be worthwhile to investigate if it would be possible to write a version of the _ssl extension that links with Apple frameworks (like CommonCrypto) instead of OpenSSL.   There are two obvious reason why this might not work out: Apple's frameworks might not over all functionality needed to implement _ssl (and _hashlib, and the additional code adds maintenance overhead that could be too high.

Antoine: Apple almost certainly has hacked their copy of OpenSSL, they do so for other libraries (including python) as well.

Apple does not ship CAcerts root certificate, I've added it to the System Keychain on my machine because I use a number of machines that use CAcert signed certificates. 

And I've found Apple's copy of the OpenSSL sources: <http://www.opensource.apple.com/source/OpenSSL/>

More interesingly are the download archives for OpenSSL098, which is the openssl version that's used on newer OSX releases. Sadly enough the version used on OSX 10.8 is not present there (that seems to be OpenSSL098-47, the latest download is -35).

Download link: <http://opensource.apple.com/tarballs/OpenSSL098/>

I've downloaded the most recent release from that link and based on a 5 minute glance at the code this does seem to integrate with the system keychain ("src/crypto/x509/x509_vfy_apple.c" in the source tee)

And that code also pointed to a way to disable that functionality, and that resolves the test failure on my machine.

Without workaround:


$ ./python.exe -m test.regrtest  -uall test_ssl
[1/1] test_ssl
test test_ssl failed -- multiple errors occurred; run in verbose mode for details
1 test failed:
    test_ssl


With workaround:

$ env OPENSSL_X509_TEA_DISABLE=1 ./python.exe -m test.regrtest  -uall test_ssl
[1/1] test_ssl
Resource 'ipv6.google.com' is not available
Warning -- asyncore.socket_map was modified by test_ssl
1 test altered the execution environment:
    test_ssl

(And test.regrtest -v also shows that all tests passed)


The attached (crufty) patch sets the environment variable during test_ssl.NetworkedTests and that also avoids the test failure. It might be  useful to add this functionality to the test case (but less crufty, and with a comment that explains why this is done).

> The attached (crufty) patch sets the environment variable during
> test_ssl.NetworkedTests and that also avoids the test failure. It might 
> be  useful to add this functionality to the test case (but less crufty, 
> and with a comment that explains why this is done).

I would prefer it to be done only under OS X platforms.

Attached cleaner version of the test:

* use self.addCleanup instead of a tearDown method
* add comment that explains why the code is present
* setUp method is only active on OSX

> Attached cleaner version of the test:
> 
> * use self.addCleanup instead of a tearDown method
> * add comment that explains why the code is present
> * setUp method is only active on OSX

Looks good to me, thank you.

This also affects 2.7, and the patch doesn't work there.

It does work when I add the call to os.putenv at module scope before (before importing ssl), but I don't really like that. 

It is probably necessary to do it like this though, the code that checks if the keychain integration is enabled caches its results and which means my patch only works accidentally (based on the order tests happen to be run in). 

I've attached a 3th version of the patch that also works with 2.7 (that is, after manually applying the patch). I'm not too happy about it though, the module now changes the environment of the entire test suite (on OSX).

Annoyingly the 3th version does *not* work with 'make test' for python 3.3 , even when I change os.environ. I haven't tried to debug that yet.

I'm disabling the CAcert root key in my keychain for no to be able to test patches without running into this issue.

Le mercredi 22 août 2012 à 11:57 +0000, Ronald Oussoren a écrit :
> I've attached a 3th version of the patch that also works with 2.7
> (that is, after manually applying the patch). I'm not too happy about
> it though, the module now changes the environment of the entire test
> suite (on OSX).

This will fail if the ssl module has already been imported by another
test.
Given the workaround is so fragile, I would recommend skipping the test
on OS X instead.

There is no need to unconditionally skip the test. The cacert.org root certificate is not present on most systems, I just happened to have imported it into my keychain.

I've removed the cacert root from my keychain and test_ssl now passed without a patch. 

I agree that is it not worthwhile to pursue this further and will therefore close this issue as "wont fix".