Fail with OpenSSL 1.1.1d  10 Sep 2019 on AMD64 FreeBSD Shared 3.x:
https://buildbot.python.org/all/#/builders/371/builds/78

======================================================================
FAIL: test_min_max_version (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1238, in test_min_max_version
    self.assertIn(
AssertionError: <TLSVersion.TLSv1_2: 771> not found in {<TLSVersion.SSLv3: 768>, <TLSVersion.TLSv1: 769>}

======================================================================
FAIL: test_min_max_version_mismatch (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 220, in wrapper
    return func(*args, **kw)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 3840, in test_min_max_version_mismatch
    self.assertIn("alert", str(e.exception))
AssertionError: 'alert' not found in '[SSL: NO_PROTOCOLS_AVAILABLE] no protocols available (_ssl.c:1108)'


SSL infos from pythoninfo:

ssl.HAS_SNI: True
ssl.OPENSSL_VERSION: OpenSSL 1.1.1d  10 Sep 2019
ssl.OPENSSL_VERSION_INFO: (1, 1, 1, 4, 15)
ssl.OP_ALL: 0x80000054
ssl.OP_NO_TLSv1_1: 0x10000000

ssl.SSLContext.maximum_version: TLSVersion.MAXIMUM_SUPPORTED
ssl.SSLContext.minimum_version: TLSVersion.MINIMUM_SUPPORTED
ssl.SSLContext.options: Options.OP_ALL|OP_NO_SSLv3|OP_CIPHER_SERVER_PREFERENCE|OP_ENABLE_MIDDLEBOX_COMPAT|OP_NO_COMPRESSION
ssl.SSLContext.protocol: _SSLMethod.PROTOCOL_TLS
ssl.SSLContext.verify_mode: VerifyMode.CERT_NONE

ssl.default_https_context.maximum_version: TLSVersion.MAXIMUM_SUPPORTED
ssl.default_https_context.minimum_version: TLSVersion.MINIMUM_SUPPORTED
ssl.default_https_context.options: Options.OP_ALL|OP_NO_SSLv3|OP_CIPHER_SERVER_PREFERENCE|OP_ENABLE_MIDDLEBOX_COMPAT|OP_NO_COMPRESSION
ssl.default_https_context.protocol: _SSLMethod.PROTOCOL_TLS
ssl.default_https_context.verify_mode: VerifyMode.CERT_REQUIRED

ssl.stdlib_context.maximum_version: TLSVersion.MAXIMUM_SUPPORTED
ssl.stdlib_context.minimum_version: TLSVersion.MINIMUM_SUPPORTED
ssl.stdlib_context.options: Options.OP_ALL|OP_NO_SSLv3|OP_CIPHER_SERVER_PREFERENCE|OP_ENABLE_MIDDLEBOX_COMPAT|OP_NO_COMPRESSION
ssl.stdlib_context.protocol: _SSLMethod.PROTOCOL_TLS
ssl.stdlib_context.verify_mode: VerifyMode.CERT_NONE

Same failure on AMD64 FreeBSD Shared 3.8:
https://buildbot.python.org/all/#/builders/374/builds/32

FAIL: test_min_max_version (test.test_ssl.ContextTests)
FAIL: test_min_max_version_mismatch (test.test_ssl.ThreadedTests)

And the last one, AMD64 FreeBSD Shared 3.7:
https://buildbot.python.org/all/#/builders/367/builds/25

FAIL: test_min_max_version (test.test_ssl.ContextTests)
FAIL: test_min_max_version_mismatch (test.test_ssl.ThreadedTests)

Similar issue in Fedora: https://src.fedoraproject.org/rpms/python3/pull-request/155

Debian and Fedora are already using OPENSSL_CONF=/non-existing-file workaround.

IMHO test_ssl should be fixed instead. test_ssl should not make assumptions on min/max TLS version.

Previous attempt:
https://github.com/python/cpython/pull/11508#issuecomment-453186969

Old Fedora issue, worked around in the Fedora buildbot configuration: bpo-35045.

I marked bpo-38954 as duplicate of this issue. Copy of its message:
---
 This issue is probably duplicate of some other, but I decided to make one new one due to the fact that this affect all Fedora build bots.


test test_ssl failed
test_timeout_connect_ex (test.test_ssl.NetworkedTests) ... ok
======================================================================
FAIL: test_min_max_version (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.8.cstratak-fedora-rawhide-x86_64.lto-pgo/build/Lib/test/test_ssl.py", line 1207, in test_min_max_version
    self.assertEqual(
AssertionError: <TLSVersion.TLSv1_3: 772> != <TLSVersion.MAXIMUM_SUPPORTED: -1>
----------------------------------------------------------------------
Ran 161 tests in 2.681s
FAILED (failures=1, skipped=11)
1 test failed again:
    test_ssl

Example failure:

https://buildbot.python.org/all/#/builders/222
---

Raising the priority to 'critical' as this is masking all other problems on a significant number of buildbots.

Christian, could you take a look?

Tomas Orsava is going to provide a PR within the next hour.

New changeset 34864d1cffdbfc620f8517dab9a68ae9a37b8c53 by Miss Islington (bot) (torsava) in branch 'master':
bpo-38815: Accept TLSv3 default in min max test (GH-NNNN) (GH-17437)
https://github.com/python/cpython/commit/34864d1cffdbfc620f8517dab9a68ae9a37b8c53

Testing for the latest PR happening here:

https://buildbot.python.org/all/#/builders?tags=%2Bstable&tags=%2Bcustom

New changeset 9e728806d03fff8fa9e75159e567b2b4e040971b by Miss Islington (bot) in branch '3.8':
bpo-38815: Accept TLSv3 default in min max test (GH-NNNN) (GH-17437)
https://github.com/python/cpython/commit/9e728806d03fff8fa9e75159e567b2b4e040971b

New changeset 1f4f28cbf3ca4c7f85b41b1f7b9455a375415cc0 by Miss Islington (bot) in branch '3.7':
bpo-38815: Accept TLSv3 default in min max test (GH-NNNN) (GH-17437)
https://github.com/python/cpython/commit/1f4f28cbf3ca4c7f85b41b1f7b9455a375415cc0

All Fedora tests are successful, so I will de-escalate the priority of the issue. Thank you very much Christian and Tomas Orsava!

test_ssl still fails on FreeBSD:

======================================================================
FAIL: test_min_max_version (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1244, in test_min_max_version
    self.assertIn(
AssertionError: <TLSVersion.TLSv1_2: 771> not found in {<TLSVersion.SSLv3: 768>, <TLSVersion.TLSv1: 769>}

======================================================================
FAIL: test_min_max_version_mismatch (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 220, in wrapper
    return func(*args, **kw)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 3846, in test_min_max_version_mismatch
    self.assertIn("alert", str(e.exception))
AssertionError: 'alert' not found in '[SSL: NO_PROTOCOLS_AVAILABLE] no protocols available (_ssl.c:1108)'

In case it's relevant, I note the following:

The ssl module is built with -I/usr/local/include in the compile line.

Leaving aside:

 - the warts of the build system, and 
 - the lack of specific --with-foo=<path> semantics to allow providing well(narrowly)-scoped include/library paths
 - how and whether the default python build using -I/usr/local/include is 'good' or not ...

it means that the build is finding openssl111 on FreeBSD provided by ports, not base (base also includes openssl 1.1.1 @ /usr/include|lib)

More specifically, our openssl111 port happens to have many options that allow enabling/disabling various software features, including encryption algorithms and protocol versions

The current (installed) build of openssl111 on the buildbot worker that is failing, has the following options set/unset:

OPTIONS_FILE_UNSET+=CT
OPTIONS_FILE_UNSET+=MAN3
OPTIONS_FILE_UNSET+=RFC3779
OPTIONS_FILE_SET+=SHARED
OPTIONS_FILE_UNSET+=ZLIB
OPTIONS_FILE_UNSET+=ARIA
OPTIONS_FILE_UNSET+=DES
OPTIONS_FILE_UNSET+=GOST
OPTIONS_FILE_UNSET+=IDEA
OPTIONS_FILE_UNSET+=SM2
OPTIONS_FILE_UNSET+=SM3
OPTIONS_FILE_UNSET+=SM4
OPTIONS_FILE_UNSET+=RC2
OPTIONS_FILE_UNSET+=RC4
OPTIONS_FILE_UNSET+=RC5
OPTIONS_FILE_UNSET+=MD2
OPTIONS_FILE_UNSET+=MD4
OPTIONS_FILE_UNSET+=MDC2
OPTIONS_FILE_UNSET+=RMD160
OPTIONS_FILE_SET+=ASM
OPTIONS_FILE_SET+=SSE2
OPTIONS_FILE_SET+=THREADS
OPTIONS_FILE_SET+=EC
OPTIONS_FILE_SET+=NEXTPROTONEG
OPTIONS_FILE_UNSET+=SCTP
OPTIONS_FILE_UNSET+=SSL3
OPTIONS_FILE_UNSET+=TLS1
OPTIONS_FILE_UNSET+=TLS1_1
OPTIONS_FILE_SET+=TLS1_2

In summary, this build only has TLS 1.2 and TLS 1.3 enabled (but with other various things disabled, not the defaults of the upstream openssl build), and may be indicated in this test failure, possibly only one example of many other similar issues of the same class, something like "tests assume certain features"

Relatedly, this method of disabling various default options in openssl, is how a number [1] of issues in the Python cryptography package were found:

https://github.com/pyca/cryptography/issues/5065

* Another issue is a build failure with NPN disabled, yet to be reported

@koobs, FWIW, unlike for most other third-party libs, there now is a configure option to allow specifying the path to the desired OpenSSL version, actually two ways.  Either use the --with-openssl= option to ./configure. Otherwise, if pkg-config info is availble, ./configure will try to use that for OpenSSL.

I had to rebuild the openssl (1.1.x) port on the worker that had test_ssl failing in order to bring it back to green, so as not to hide new test failures.

If/when someone is able to produce a fix for the failing test, I am happy to revert the openssl build to its prior (failing config) state at any time to verify the test fix.

@Christian As current assignee, are you able to produce a test that fixes the remaining issue (per msg357792)? I can rebuild OpenSSL on the worker at your direction at any time to make the test fail again or provide you with an SSH account to assist

I'm not sure why, but test_ssl does not pass on FreeBSD and Fedora, so I close the issue.

Please don't close tickets assigned to an owner without consent.

> Please don't close tickets assigned to an owner without consent.

Can you please elaborate why you reopened the issue?

The initial "test_ssl: test_min_max_version() fails on FreeBSD and Fedora" issue is now fixed.

For example, test_ssl pass on AMD64 Fedora Stable LTO 3.x buildbot and it no longer uses the OPENSSL_CONF=/non-existing-file workaround.
https://buildbot.python.org/all/#/builders/112/builds/850
"ssl.OPENSSL_VERSION: OpenSSL 1.1.1g FIPS  21 Apr 2020"

Same for AMD64 FreeBSD Shared 3.x:
https://buildbot.python.org/all/#/builders/152/builds/852
"ssl.OPENSSL_VERSION: OpenSSL 1.1.1g  21 Apr 2020"

If there are remaining new issues, I would prefer to open new issues.

If you think that a ticket no longer apply, please use the pending status and give me a chance to verify the result. I haven't looked into the issue and I don't have time to do it right now.

Ping?

still getting this issue on Ubuntu 20.04 with the latest master checkout of cpython

I haven't seen the problem in a while.