FAIL: test_min_max_version_mismatch (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vbk/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 217, in wrapper
    return func(*args, **kw)
  File "/home/vbk/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 3841, in test_min_max_version_mismatch
    self.assertIn("alert", str(e.exception))
AssertionError: 'alert' not found in '[SSL: NO_PROTOCOLS_AVAILABLE] no protocols available (_ssl.c:1123)'

Has any progress been made on the Ubuntu 20.04 test_ssl failures? Is there any consensus about it being a Python or Ubuntu problem?

This skips the breaking tests (but doesn't actually fix anything).

I don't know if it matters, but I started having this problem when I switched from Ubuntu 18.04 (native python3.7) to 20.04 (native python3.8.2). I specify --prefix to a folder in my home directory, but while running make test Ubuntu gives a system error which refers to Ubuntu's python. I don't know exactly at what test it happens, approximately in the middle, but it should not happen at all because the tests should only call the python compiled by me.

This will help to solve it

https://stackoverflow.com/questions/61568215/openssl-v1-1-1-ubuntu-20-tlsv1-no-protocols-available

But in my case I've defined:
MinProtocol = None

@Vladyslav.Bondar I can't tell where you are suggesting MinProtocol should be set. I don't see that particular string in any .c, .h or .py file in the Python source.

This is about openssl configuration in Ubuntu. In the latest Ubuntu, they disabled TLS 1.0/1.1.

So to enable it back there is a workaround (taken from StackOverflow):

You should modify openssl config: /etc/ssl/openssl.cnf

You need to add this to the beginning of your config file:

openssl_conf = default_conf

And then this to the end:

[ default_conf ]

ssl_conf = ssl_sect

[ssl_sect]

system_default = ssl_default_sect

[ssl_default_sect]
MinProtocol = None
CipherString = DEFAULT:@SECLEVEL=1

issue38815 also reported similar issue in test_min_max_version_mismatch.

I followed Vladyslav Bondar's 2020-09-11 09:10:30 recommendations and it worked:
Tests result: SUCCESS

Thank you.

It is not clear though how Canonical built its python-3.8.2 which comes with Ubuntu-20.04. Does anyone know someone at Canonical to ask this question?

Can test_ssl script determine which TLS versions are enabled in a particular Linux distribution and run tests only for enabled versions?

I just ran into this too on Ubuntu 20.04.

I don't recommend changing /etc/ssl/openssl.cnf. Instead, make a local copy, for example at $HOME/cpython-dev-openssl.cnf, with the changes suggested by Vladyslav Bondar. Then run the tests with:

OPENSSL_CONF=$HOME/openssl-cpython-dev.cnf ./python -m test

I confirm that Tal Einat's workaround also works.

>
> I confirm that Tal Einat's workaround also works.
>

Should workarounds be required to successfully run the test suite? I
routinely unset PYTHONSTARTUP, but that's because I can and sometimes do
weird things to support interactive use. It concerns me a bit that any
system settings need to be overridden. That suggests to me that it will
eventually bite users at runtime.

>

I think it is not a real solution, just a workaround. What do you think the solution should be?

Could you please modify the test case to check for TLS 1.0 and run it Ubuntu?

    @requires_minimum_version
    @requires_tls_version('TLSv1_2')
    @requires_tls_version('TLSv1')
    def test_min_max_version_mismatch(self):

For Python 3.10 I'm planning to drop support for TLS 1.1 and earlier.

> Could you please modify the test case to check for TLS 1.0 and run it Ubuntu?
>
>     @requires_minimum_version
>     @requires_tls_version('TLSv1_2')
>     @requires_tls_version('TLSv1')
>     def test_min_max_version_mismatch(self):

Confirmed. test_min_max_version_mismatch passes with those three decorators.

On my system, it still fails.

Here is the content of modified Lib/test/test_ssl.py:

3827     @requires_minimum_version
3828     @requires_tls_version('TLSv1_2')
3829     @requires_tls_version('TLSv1')
3830     def test_min_max_version_mismatch(self):
3831         client_context, server_context, hostname = testing_context()
3832         # client 1.0, server 1.2 (mismatch)
3833         server_context.maximum_version = ssl.TLSVersion.TLSv1_2
3834         server_context.minimum_version = ssl.TLSVersion.TLSv1_2
3835         client_context.maximum_version = ssl.TLSVersion.TLSv1
3836         client_context.minimum_version = ssl.TLSVersion.TLSv1
3837         with ThreadedEchoServer(context=server_context) as server:
3838             with client_context.wrap_socket(socket.socket(),
3839                                             server_hostname=hostname) as s:
3840                 with self.assertRaises(ssl.SSLError) as e:
3841                     s.connect((HOST, server.port))
3842                 self.assertIn("alert", str(e.exception))
3843
3844     @requires_minimum_version
3845     @requires_tls_version('SSLv3')

https://www.python.org/ftp/python/3.8.5/Python-3.8.5.tar.xz

...
======================================================================
ERROR: test_protocol_tlsv1 (test.test_ssl.ThreadedTests)
Connecting to a TLSv1 server with various client options
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 217, in wrapper
    return func(*args, **kw)
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 3286, in test_protocol_tlsv1
    try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_TLSv1, 'TLSv1')
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 2780, in try_protocol_combo
    stats = server_params_test(client_context, server_context,
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 2695, in server_params_test
    s.connect((HOST, server.port))
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/ssl.py", line 1342, in connect
    self._real_connect(addr, False)
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/ssl.py", line 1333, in _real_connect
    self.do_handshake()
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL] internal error (_ssl.c:1123)

======================================================================
ERROR: test_protocol_tlsv1_1 (test.test_ssl.ThreadedTests)
Connecting to a TLSv1.1 server with various client options.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 217, in wrapper
    return func(*args, **kw)
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 3302, in test_protocol_tlsv1_1
    try_protocol_combo(ssl.PROTOCOL_TLSv1_1, ssl.PROTOCOL_TLSv1_1, 'TLSv1.1')
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 2780, in try_protocol_combo
    stats = server_params_test(client_context, server_context,
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 2695, in server_params_test
    s.connect((HOST, server.port))
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/ssl.py", line 1342, in connect
    self._real_connect(addr, False)
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/ssl.py", line 1333, in _real_connect
    self.do_handshake()
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL] internal error (_ssl.c:1123)

======================================================================
FAIL: test_min_max_version_mismatch (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 217, in wrapper
    return func(*args, **kw)
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 217, in wrapper
    return func(*args, **kw)
  File "/home/bugrep/Downloads/Python-3.8.5/Lib/test/test_ssl.py", line 3842, in test_min_max_version_mismatch
    self.assertIn("alert", str(e.exception))
AssertionError: 'alert' not found in '[SSL: NO_PROTOCOLS_AVAILABLE] no protocols available (_ssl.c:1123)'

It's starting to look like a misconfiguration in either Ubuntu's OpenSSL build or your system. has_tls_version() checks compile time options and runtime configuration options. It should detect that TLS 1.1 and 1.0 are not available. "[SSL] internal error" also points to an unusual error condition that should never be triggered by these tests.

Please run this on your system:

import ssl
print((ssl.HAS_TLSv1, ssl.HAS_TLSv1_1, ssl.HAS_TLSv1_2, ssl.HAS_TLSv1_3))
print(ssl.SSLContext().maximum_version)
print(ssl.SSLContext().minimum_version)

(True, True, True, True)
TLSVersion.MAXIMUM_SUPPORTED
TLSVersion.MINIMUM_SUPPORTED

In case it's needed:
TLSVersion.MAXIMUM_SUPPORTED == -1
TLSVersion.MINIMUM_SUPPORTED == -2

Likewise here on Ubuntu 20.04:

(True, True, True, True)
-1
-2

Four times 'True' means that OpenSSL is compiled with TLS 1.0, 1,1, 1.2, and 1.3 support. SSLContext().minium_version == MINIMUM_SUPPORTED and maximum_version == MAXIMUM_SUPPORTED mean that no crypto policy setting or OpenSSL security level setting has modified the minimum and maximum version. TLS 1.0 and 1.1 connection should work.

But it's not working for you and some connection are even failing "internal error". This smells like a Debian/Ubuntu have patched OpenSSL with a buggy patch that breaks OpenSSL's internal API.

Does "test_min_max_version_mismatch" pass on your system when you add

    client_context.set_ciphers("@SECLEVEL=1:HIGH")

?

No, adding that after the first line of test_min_max_version_mismatch() still results in the same error:

ssl.SSLError: [SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1122)

What happens if you modify both contexts?

Same.

I added client_context.set_ciphers("@SECLEVEL=1:HIGH"), then added server_context.set_ciphers("@SECLEVEL=1:HIGH"). The test failed in both cases.

I did not have problem with python 3.7.x. in Ubuntu 18.04. I have just tried compiling 3.7.5 in Ubuntu 20.04 and test_ssl failed.

I also remember downloading openssl source code, compiling it, and using --with-openssl=DIR option with python 3.8.x in Ubuntu 20.04. I tried different versions of openssl (I did not edit any config files, just compiled) and test_ssl failed with all of them. Does it mean that Ubuntu's config files were still used even in this case?

Did you set an rpath, reconfigure ldconfig, or LD_LIBRARY_PATH env var? If not then you compiled Python with an alternative OpenSSL installation but did not instruct the ld to load the alternative shared libraries. --with-openssl only modifies header and linker search, not dynamic loader options.

The command

    ldd $(find build -name '_ssl*.so')

will should you what shared OpenSSL libraries the dynamic linker will load.

I wrote a script to download and compile OpenSSL and then run Python's test suite with exactly that OpenSSL build:

    ./configure
    make
    ./python Tools/ssl/multissltests.py --openssl=1.1.1g

1) I downloaded openssl-1.1.1g source code, compiled, exported LD_LIBRARY_PATH, did sudo ldconfig - ./python Lib/test/test_ssl.py now passes. If LD_LIBRARY_PATH is unset, it fails. Thank you for advice.


2) Tried the following independently from (1)
./configure
make
./python Tools/ssl/multissltests.py --openssl=1.1.1g

== Tests result: SUCCESS ==

All 15 tests OK.

Total duration: 1 min 21 sec
Tests result: SUCCESS
*** INFO 
Tests finished in 0:05:52.488541
Python:  3.8.5 (default, Sep 21 2020, 23:02:31) 
[GCC 9.3.0]
Executed all SSL tests.
OpenSSL / LibreSSL versions:
    * OpenSSL 1.1.1g

Is it worth comparing openssl vanilla code and configs with Ubuntu's version?

Yes, that would be useful. I suspect tls1.2-min-seclevel2.patch from the patch set https://launchpad.net/ubuntu/+source/openssl/1.1.1f-1ubuntu2 might be the cause of this issue.

I downloaded Ubuntu's openssl_1.1.1f.orig.tar.gz and openssl_1.1.1f-1ubuntu2.debian.tar.xz from https://launchpad.net/ubuntu/+source/openssl/1.1.1f-1ubuntu2, but I did not know how to apply patches. In addition, too many files differ, so I could not understand what makes test_ssl fail. So I took a different approach.

In Ubuntu-20.04, "apt policy openssl" returned the version of the installed library: 1.1.1f-1ubuntu2. I ran "apt source openssl" to downloade the source code from Ubuntu. I compiled, tested and installed it.


If LD_LIBRARY_PATH is not set, ldd returns this:
  libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1
  libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1

make test TESTOPTS="-v test_ssl":
  FAILED (errors=6, skipped=11)


If LD_LIBRARY_PATH is set to compiled openssl-1.1.1f-1ubuntu2:
  libssl.so.1.1 => /home/bugsrep/openssl-ubuntu2/lib/libssl.so.1.1 
  libcrypto.so.1.1 => /home/bugsrep/openssl-ubuntu2/lib/libcrypto.so.1.1

make test TESTOPTS="-v test_ssl":
  OK (skipped=11)
  == Tests result: SUCCESS ==
  1 test OK.

Please note that test_ssl also passes if /etc/ssl/openssl.conf is modified per msg376705 by Vladyslav Bondar (with /usr/lib/x86_64-linux-gnu/libssl.so.1.1)

Just tested python 3.9.0 - same issue.

It sounds like a Debian/Ubuntu patch is breaking an assumption. Did somebody report the bug with Debian/Ubuntu maintainers of OpenSSL already?

Fedora also configures OpenSSL with minimum protocol version of TLS 1.2. The distribution does it in a slightly different way that makes the restriction discoverable and that is compatible with Python's test suite.

I started by asking a question at https://askubuntu.com/questions/1281942/pythons-test-ssl-fails-starting-from-ubuntu-20-04-i-need-to-find-a-person-at-c

I got an advice and posted the question at https://answers.launchpad.net/ubuntu/+source/openssl/+question/693423

I reported a bug at https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1899878

New changeset ce04e7105bc396c32667a22b928a712ba0778a3f by Christian Heimes in branch 'master':
bpo-41561: skip test_min_max_version_mismatch (GH-22308)
https://github.com/python/cpython/commit/ce04e7105bc396c32667a22b928a712ba0778a3f

New changeset 73e02ff0d47c37cf2a8f137cfbea0b36d26c48bb by Miss Islington (bot) in branch '3.8':
bpo-41561: skip test_min_max_version_mismatch (GH-22308)
https://github.com/python/cpython/commit/73e02ff0d47c37cf2a8f137cfbea0b36d26c48bb

New changeset 802ff7c0d339376a1b974e57d2caca898310de3d by Miss Islington (bot) in branch '3.9':
[3.9] bpo-41561: skip test_min_max_version_mismatch (GH-22308) (GH-23363)
https://github.com/python/cpython/commit/802ff7c0d339376a1b974e57d2caca898310de3d

Christian, I don't see any open PRs to be commit reviewed.

Downstream has asked me to file a separate bug for internal error during handshake. The problem is tracked at https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1917625 .

I have discussed the problem with downstream engineers on the two issues

- https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1899878
- https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1917625

The gist of the issue is: Canonical has taken a different approach than Debian and other distros to set minimum TLS version.

Most distros use an openssl.cnf file to set "MinProtocol = TLSv1.2". The config file approach allows application to override the setting with SSL_CTX_set_min_proto_version(ctx, TLS1_VERSION) and to detect the current minimum version with SSL_CTX_get_min_proto_version(ctx) == TLS1_VERSION.

Ubuntu doesn't set "MinProtocol = TLSv1.2". Instead the distro has patched OpenSSL source code and modified the meaning of security level "2". Security level is a new OpenSSL API to set various security related settings. On Ubuntu SECLEVEL=2 prevents TLS 1.0 and 1.1 connection. Further SSL_CTX_get_min_proto_version(ctx) returns 0 (dummy value for minimum supported version). SSL_CTX_set_min_proto_version(ctx, TLS1_VERSION) does not fail although TLS 1.0 is prohibited.

https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_security_level.html
    Level 2: SSL version 3 is also not allowed
    Level 4: TLS versions below 1.2 are not permitted.

https://manpages.ubuntu.com/manpages/focal/man3/SSL_CTX_set_security_level.3ssl.html
    Level 2: On Ubuntu, TLS versions below 1.2 are not permitted

The combination of "Ubuntu changed the meaning of security level policy" and "SSL_CTX_get_min_proto_version(ctx) does not report minimum version" breaks our tests.

OpenSSL doesn't provide an easy way to check if a SSL_CTX has a sane configuration. There is a way to check if a security policy allows a TLS version. I'm not sure if we should include the check in CPython and where to best put the check:

    void *sec_ex = SSL_CTX_get0_security_ex_data(ctx);
    sec_cb = SSL_CTX_get_security_callback(ctx);
    int result = sec_cb(NULL, ctx, SSL_SECOP_VERSION, 0, TLS1_VERSION, NULL, sec_ex);
    if (result && (SSL_CTX_get_min_proto_version(ctx) >=  TLS1_VERSION)) ...

Dimitri John Ledkov from Canonical has opened a feature request for a context validation feature on the OpenSSL issue tracker, https://github.com/openssl/openssl/issues/14607

New changeset f6c6b5821bff815bdc810de53992fd1fbdb2edd4 by Christian Heimes in branch 'master':
bpo-41561: Add workaround for Ubuntu's custom security level (GH-24915)
https://github.com/python/cpython/commit/f6c6b5821bff815bdc810de53992fd1fbdb2edd4

New changeset 3365e684a83a6bc9e2e2198dca54b42711bd3c90 by Miss Islington (bot) in branch '3.9':
bpo-41561: Add workaround for Ubuntu's custom security level (GH-24915)
https://github.com/python/cpython/commit/3365e684a83a6bc9e2e2198dca54b42711bd3c90

New changeset 50511677f59464e612cfef0cd0e139fe07e87737 by Miss Islington (bot) in branch '3.8':
bpo-41561: Add workaround for Ubuntu's custom security level (GH-24915)
https://github.com/python/cpython/commit/50511677f59464e612cfef0cd0e139fe07e87737

New changeset 3447750073aff229b049e4ccd6217db2811dcfd1 by Christian Heimes in branch 'master':
bpo-41561: Fix testing with OpenSSL 1.0.2 (GH-25355)
https://github.com/python/cpython/commit/3447750073aff229b049e4ccd6217db2811dcfd1

New changeset 04425a922b598d03770619b0c658ee9874113550 by Miss Islington (bot) in branch '3.8':
bpo-41561: Fix testing with OpenSSL 1.0.2 (GH-25355)
https://github.com/python/cpython/commit/04425a922b598d03770619b0c658ee9874113550

New changeset 0983e01837714524fb164e784a8e96a2bc4bdf94 by Miss Islington (bot) in branch '3.9':
bpo-41561: Fix testing with OpenSSL 1.0.2 (GH-25355)
https://github.com/python/cpython/commit/0983e01837714524fb164e784a8e96a2bc4bdf94

New changeset 64be96ae1f85ce6b3bca4328576cf62d73f77b2a by Christian Heimes in branch '3.7':
[3.7] bpo-41561: Add workaround for Ubuntu's custom security level (GH-24915) (GH-24928)
https://github.com/python/cpython/commit/64be96ae1f85ce6b3bca4328576cf62d73f77b2a

New changeset 512742d554f2c10e9a273855d87a68f5ee93ed29 by Miss Islington (bot) in branch '3.7':
bpo-41561: Fix testing with OpenSSL 1.0.2 (GH-25355) (GH-25858)
https://github.com/python/cpython/commit/512742d554f2c10e9a273855d87a68f5ee93ed29