classification
Title: Make Python compatible with OpenSSL 3.0.0
Type: enhancement Stage: patch review
Components: SSL Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: 43788 43789 43791 43794 43799 43811 Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, cstratak, hroncok, iritkatriel, miss-islington
Priority: high Keywords: patch

Created on 2019-11-16 15:06 by christian.heimes, last changed 2021-04-13 07:08 by christian.heimes.

Pull Requests
URL Status Linked Edit
PR 17190 merged christian.heimes, 2019-11-16 15:11
PR 17499 merged miss-islington, 2019-12-07 16:59
PR 17500 merged miss-islington, 2019-12-07 16:59
PR 25316 closed christian.heimes, 2021-04-09 17:59
PR 25319 merged christian.heimes, 2021-04-09 20:07
PR 25320 merged miss-islington, 2021-04-09 20:23
PR 25321 merged miss-islington, 2021-04-09 20:23
Messages (12)
msg356750 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-11-16 15:06
OpenSSL 3.0.0 is currently development [1]. I'm expecting a first beta release in December. Final release is scheduled for Q2 2020. OpenSSL 3.0.0 is API and feature compatible to OpenSSL 1.1.0 and 1.1.1. Only minor changes are required:

* OpenSSL version number is >= 3.0.0, which breaks test_openssl_version
* GENERAL_NAME_print() no longer adds trailing newline to IPv6 address strings. 
* ERR_func_error_string is deprecated

[1] https://www.openssl.org/blog/blog/2019/11/07/3.0-update/
msg356759 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-11-16 17:22
PR GH-17190 fixes test_openssl_version and removes the trailing newline from IPv6 addresses on all OpenSSL versions. I prefer to have the output consistent on all OpenSSL versions. The newline was silly any way.
msg357979 - (view) Author: miss-islington (miss-islington) Date: 2019-12-07 16:59
New changeset 2b7de6696bf2f924cd2cd9ff0a539c8aa37c6244 by Miss Islington (bot) (Christian Heimes) in branch 'master':
bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190)
https://github.com/python/cpython/commit/2b7de6696bf2f924cd2cd9ff0a539c8aa37c6244
msg357980 - (view) Author: miss-islington (miss-islington) Date: 2019-12-07 17:20
New changeset 9d3cacd5901f8fbbc4f8b78fc35abad01a0e6546 by Miss Islington (bot) in branch '3.8':
[3.8] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) (GH-17499)
https://github.com/python/cpython/commit/9d3cacd5901f8fbbc4f8b78fc35abad01a0e6546
msg357981 - (view) Author: miss-islington (miss-islington) Date: 2019-12-07 17:20
New changeset a197f8aa7493e66bc54c3db8f796d00cef1c3042 by Miss Islington (bot) in branch '3.7':
[3.7] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) (GH-17500)
https://github.com/python/cpython/commit/a197f8aa7493e66bc54c3db8f796d00cef1c3042
msg378790 - (view) Author: Irit Katriel (iritkatriel) * (Python triager) Date: 2020-10-16 23:24
Can this be closed?
msg378807 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-10-17 11:11
No, this is still work in progress.
msg390485 - (view) Author: Miro HronĨok (hroncok) * Date: 2021-04-07 21:59
Python 3.10.0a7 with OpenSSL 3.0 from https://copr.fedorainfracloud.org/coprs/saprasad/openssl-3.0/ in https://copr.fedorainfracloud.org/coprs/g/python/openssl-3.0/package/python3.10/ (full logs available there).

3 tests failed:
    test_imaplib test_ssl test_urllib2_localnet

Many:

ssl.SSLError: [SSL: KRB5_S_TKT_NYV] unexpected eof while reading (_ssl.c:2628)

Also:

Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.10.0a7/Lib/test/test_ssl.py", line 1413, in test_load_cert_chain
    ctx.load_cert_chain(CERTFILE_PROTECTED, password=getpass_huge)
SystemError: _PyEval_EvalFrameDefault returned a result with an exception set

And:

 ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1122)

 ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:1122)

 ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1122)
msg390650 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-09 17:19
Miro,

I have pushed several fixes for OpenSSL 3.0.0

* bpo-43788 addresses wrong library and error reason codes (e.g. KRB5_S_TKT_NYV)
* bpo-43789 fixes an issue with exception state in password callbacks (_PyEval_EvalFrameDefault returned a result with an exception set)
* bpo-43791 disables TLS 1.0 and 1.1 testing with OpenSSL 3.0.0. I'll have to talk to upstream and figure out a better solution.
* bpo-43794 adds OP_IGNORE_UNEXPECTED_EOF and sets it by default. This makes the code behave like OpenSSL 1.1.0 and 1.0.2.

I'll look into the other issues next week.
msg390662 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-09 20:23
New changeset 2d7fdc90731e132f9d6b43852ee112f25831394b by Christian Heimes in branch 'master':
bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
https://github.com/python/cpython/commit/2d7fdc90731e132f9d6b43852ee112f25831394b
msg390664 - (view) Author: miss-islington (miss-islington) Date: 2021-04-09 20:33
New changeset ffb05bbb30fa82dbe887981bdabd65af7daffcd1 by Miss Islington (bot) in branch '3.8':
bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
https://github.com/python/cpython/commit/ffb05bbb30fa82dbe887981bdabd65af7daffcd1
msg390666 - (view) Author: miss-islington (miss-islington) Date: 2021-04-09 20:46
New changeset 7c8796a750fb108be99e0bc50ca3dba000d77e54 by Miss Islington (bot) in branch '3.9':
bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
https://github.com/python/cpython/commit/7c8796a750fb108be99e0bc50ca3dba000d77e54
History
Date User Action Args
2021-04-13 07:08:55christian.heimessetdependencies: + OpenSSL 3.0.0: define OPENSSL_API_COMPAT 1.1.1, Run GHA CI with multiple OpenSSL versions
2021-04-09 20:46:25miss-islingtonsetmessages: + msg390666
2021-04-09 20:33:36miss-islingtonsetmessages: + msg390664
2021-04-09 20:23:51miss-islingtonsetpull_requests: + pull_request24056
2021-04-09 20:23:40miss-islingtonsetpull_requests: + pull_request24055
2021-04-09 20:23:19christian.heimessetmessages: + msg390662
2021-04-09 20:07:23christian.heimessetpull_requests: + pull_request24054
2021-04-09 17:59:38christian.heimessetstage: patch review
pull_requests: + pull_request24051
2021-04-09 17:19:52christian.heimessetmessages: + msg390650
2021-04-09 15:19:17christian.heimessetdependencies: + OpenSSL 3.0.0: Make ssl_data.h version specific, OpenSSL 3.0.0: Handle UNEXPECTED_EOF_WHILE_READING / wrap SSL_OP_IGNORE_UNEXPECTED_EOF
2021-04-09 13:28:23christian.heimessetdependencies: + OpenSSL 3.0.0: password callback called multiple times, OpenSSL 3.0.0: TLS 1.0 / 1.1 connections fail with TLSV1_ALERT_INTERNAL_ERROR
versions: - Python 3.7
2021-04-08 00:45:51yan12125setnosy: - yan12125
2021-04-07 23:57:35cstrataksetnosy: + cstratak
2021-04-07 21:59:10hroncoksetnosy: + hroncok

messages: + msg390485
versions: + Python 3.10
2020-12-15 17:05:33yan12125setnosy: + yan12125
2020-10-17 11:11:01christian.heimessetstatus: closed -> open
resolution: fixed ->
messages: + msg378807

stage: resolved -> (no value)
2020-10-17 01:29:10methanesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-10-16 23:24:10iritkatrielsetnosy: + iritkatriel
messages: + msg378790
2019-12-07 17:20:46miss-islingtonsetmessages: + msg357981
2019-12-07 17:20:31miss-islingtonsetmessages: + msg357980
2019-12-07 16:59:55miss-islingtonsetpull_requests: + pull_request16978
2019-12-07 16:59:49miss-islingtonsetpull_requests: + pull_request16977
2019-12-07 16:59:40miss-islingtonsetnosy: + miss-islington
messages: + msg357979
2019-11-16 17:22:08christian.heimessetmessages: + msg356759
2019-11-16 15:11:57christian.heimessetkeywords: + patch
stage: patch review
pull_requests: + pull_request16695
2019-11-16 15:06:59christian.heimescreate