Make Python compatible with OpenSSL 3.0.0 #83001

tiran · 2019-11-16T15:07:00Z

BPO	38820
Nosy	@tiran, @mcepl, @ambv, @stratakis, @hroncok, @miss-islington, @iritkatriel
PRs	bpo-38820: OpenSSL 3.0.0 compatibility. #17190 [3.8] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) #17499 [3.7] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) #17500 bpo-38820: Run CI with OpenSSL 3.0.0-alpha14 #25316 bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319) #25319 [3.9] bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319) #25320 [3.8] bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319) #25321 bpo-38820: Test with OpenSSL 3.0.0-alpha15 (GH-25537) #25537 bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624) #25624 [3.9] bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624) #25626 [3.8] bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624) #25627 bpo-38820: Add ssl, hashlib, and hmac changes to whatsnew 3.10 (GH-25817) #25817 bpo-38820: Test with OpenSSL 3.0.0-alpha16 (GH-25942) #25942 [3.10] bpo-38820: Test with OpenSSL 3.0.0-alpha16 (GH-25942) #25943 [3.9] bpo-38820: Test with OpenSSL 3.0.0-alpha16 (GH-25942) #25944 bpo-38820: Test with OpenSSL 3.0.0-alpha17 #26266 [3.10] bpo-38820: Test with OpenSSL 3.0.0-alpha17 (GH-26266) #26269 bpo-38820: Test with OpenSSL 3.0.0-beta1 (GH-26769) #26769 [3.10] bpo-38820: Test with OpenSSL 3.0.0-beta1 (GH-26769) #26799 bpo-38820: Test with OpenSSL 3.0.0 final #28205 [3.10] bpo-38820: Test with OpenSSL 3.0.0 final (GH-28205) #28216 [3.9] bpo-38820: Test with OpenSSL 3.0.0 final (GH-28205) #28217
Dependencies	bpo-43788: OpenSSL 3.0.0: Make ssl_data.h version specific bpo-43789: OpenSSL 3.0.0: password callback called multiple times bpo-43791: OpenSSL 3.0.0: TLS 1.0 / 1.1 connections fail with TLSV1_ALERT_INTERNAL_ERROR bpo-43794: OpenSSL 3.0.0: Handle UNEXPECTED_EOF_WHILE_READING / wrap SSL_OP_IGNORE_UNEXPECTED_EOF bpo-43799: OpenSSL 3.0.0: define OPENSSL_API_COMPAT 1.1.1 bpo-43811: Run GHA CI with multiple OpenSSL versions bpo-43920: OpenSSL 3.0.0: handle empty cadata consistently

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = None
created_at = <Date 2019-11-16.15:06:59.538>
labels = ['expert-SSL', 'type-feature', '3.9', '3.10', '3.11']
title = 'Make Python compatible with OpenSSL 3.0.0'
updated_at = <Date 2021-09-08.17:05:14.504>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2021-09-08.17:05:14.504>
actor = 'lukasz.langa'
assignee = 'christian.heimes'
closed = False
closed_date = None
closer = None
components = ['SSL']
creation = <Date 2019-11-16.15:06:59.538>
creator = 'christian.heimes'
dependencies = ['43788', '43789', '43791', '43794', '43799', '43811', '43920']
files = []
hgrepos = []
issue_num = 38820
keywords = ['patch']
message_count = 27.0
messages = ['356750', '356759', '357979', '357980', '357981', '378790', '378807', '390485', '390650', '390662', '390664', '390666', '391688', '391887', '391892', '391940', '392701', '393098', '394031', '396120', '396123', '397326', '401311', '401355', '401407', '401408', '401409']
nosy_count = 8.0
nosy_names = ['christian.heimes', 'mcepl', 'lukasz.langa', 'cstratak', 'hroncok', 'miss-islington', 'iritkatriel', 'bweeks']
pr_nums = ['17190', '17499', '17500', '25316', '25319', '25320', '25321', '25537', '25624', '25626', '25627', '25817', '25942', '25943', '25944', '26266', '26269', '26769', '26799', '28205', '28216', '28217']
priority = 'high'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue38820'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

tiran · 2019-11-16T15:06:59Z

OpenSSL 3.0.0 is currently development [1]. I'm expecting a first beta release in December. Final release is scheduled for Q2 2020. OpenSSL 3.0.0 is API and feature compatible to OpenSSL 1.1.0 and 1.1.1. Only minor changes are required:

OpenSSL version number is >= 3.0.0, which breaks test_openssl_version
GENERAL_NAME_print() no longer adds trailing newline to IPv6 address strings.
ERR_func_error_string is deprecated

[1] https://www.openssl.org/blog/blog/2019/11/07/3.0-update/

tiran · 2019-11-16T17:22:09Z

PR #61392 fixes test_openssl_version and removes the trailing newline from IPv6 addresses on all OpenSSL versions. I prefer to have the output consistent on all OpenSSL versions. The newline was silly any way.

miss-islington · 2019-12-07T16:59:40Z

New changeset 2b7de66 by Miss Islington (bot) (Christian Heimes) in branch 'master':
bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190)
2b7de66

miss-islington · 2019-12-07T17:20:31Z

New changeset 9d3cacd by Miss Islington (bot) in branch '3.8':
[3.8] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) (GH-17499)
9d3cacd

miss-islington · 2019-12-07T17:20:46Z

New changeset a197f8a by Miss Islington (bot) in branch '3.7':
[3.7] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) (GH-17500)
a197f8a

iritkatriel · 2020-10-16T23:24:10Z

Can this be closed?

tiran · 2020-10-17T11:11:02Z

No, this is still work in progress.

hroncok · 2021-04-07T21:59:10Z

Python 3.10.0a7 with OpenSSL 3.0 from https://copr.fedorainfracloud.org/coprs/saprasad/openssl-3.0/ in https://copr.fedorainfracloud.org/coprs/g/python/openssl-3.0/package/python3.10/ (full logs available there).

3 tests failed:
test_imaplib test_ssl test_urllib2_localnet

Many:

ssl.SSLError: [SSL: KRB5_S_TKT_NYV] unexpected eof while reading (_ssl.c:2628)

Also:

Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.10.0a7/Lib/test/test_ssl.py", line 1413, in test_load_cert_chain
    ctx.load_cert_chain(CERTFILE_PROTECTED, password=getpass_huge)
SystemError: _PyEval_EvalFrameDefault returned a result with an exception set

And:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1122)

ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:1122)

ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1122)

tiran · 2021-04-09T17:19:53Z

Miro,

I have pushed several fixes for OpenSSL 3.0.0

bpo-43788 addresses wrong library and error reason codes (e.g. KRB5_S_TKT_NYV)
bpo-43789 fixes an issue with exception state in password callbacks (_PyEval_EvalFrameDefault returned a result with an exception set)
bpo-43791 disables TLS 1.0 and 1.1 testing with OpenSSL 3.0.0. I'll have to talk to upstream and figure out a better solution.
bpo-43794 adds OP_IGNORE_UNEXPECTED_EOF and sets it by default. This makes the code behave like OpenSSL 1.1.0 and 1.0.2.

I'll look into the other issues next week.

tiran · 2021-04-09T20:23:20Z

New changeset 2d7fdc9 by Christian Heimes in branch 'master':
bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
2d7fdc9

miss-islington · 2021-04-09T20:33:37Z

New changeset ffb05bb by Miss Islington (bot) in branch '3.8':
bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
ffb05bb

miss-islington · 2021-04-09T20:46:25Z

New changeset 7c8796a by Miss Islington (bot) in branch '3.9':
bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
7c8796a

tiran · 2021-04-23T12:19:24Z

New changeset dcf6581 by Christian Heimes in branch 'master':
bpo-38820: Test with OpenSSL 3.0.0-alpha15 (GH-25537)
dcf6581

tiran · 2021-04-26T08:54:20Z

New changeset 3c586ca by Christian Heimes in branch 'master':
bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624)
3c586ca

miss-islington · 2021-04-26T09:35:40Z

New changeset 10ee266 by Miss Islington (bot) in branch '3.8':
[3.8] bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624) (GH-25627)
10ee266

miss-islington · 2021-04-26T15:12:47Z

New changeset 3b917d1 by Miss Islington (bot) in branch '3.9':
[3.9] bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624) (GH-25626)
3b917d1

tiran · 2021-05-02T14:38:22Z

New changeset d8389e3 by Christian Heimes in branch 'master':
bpo-38820: Add ssl, hashlib, and hmac changes to whatsnew 3.10 (GH-25817)
d8389e3

miss-islington · 2021-05-06T14:53:19Z

New changeset f8778f9 by Miss Islington (bot) in branch '3.10':
bpo-38820: Test with OpenSSL 3.0.0-alpha16 (GH-25942)
f8778f9

miss-islington · 2021-05-20T15:11:02Z

New changeset 36843f7 by Miss Islington (bot) in branch '3.10':
bpo-38820: Test with OpenSSL 3.0.0-alpha17 (GH-26266)
36843f7

tiran · 2021-06-19T09:08:48Z

New changeset 44fb551 by Christian Heimes in branch 'main':
bpo-38820: Test with OpenSSL 3.0.0-beta1 (GH-26769)
44fb551

miss-islington · 2021-06-19T10:45:10Z

New changeset c6cd2ec by Miss Islington (bot) in branch '3.10':
[3.10] bpo-38820: Test with OpenSSL 3.0.0-beta1 (GH-26769) (GH-26799)
c6cd2ec

ambv · 2021-07-12T15:12:42Z

New changeset c92b391 by Christian Heimes in branch '3.9':
[3.9] bpo-38820: Test with OpenSSL 3.0.0-alpha16 (GH-25942) (bpo-25944)
c92b391

ambv · 2021-09-07T17:05:03Z

New changeset cc7c680 by Christian Heimes in branch 'main':
bpo-38820: Test with OpenSSL 3.0.0 final (GH-28205)
cc7c680

miss-islington · 2021-09-08T08:26:14Z

New changeset 2fe15db by Miss Islington (bot) in branch '3.10':
bpo-38820: Test with OpenSSL 3.0.0 final (GH-28205)
2fe15db

ambv · 2021-09-08T17:01:38Z

New changeset 7a6178a by Łukasz Langa in branch '3.9':
[3.9] bpo-38820: Test with OpenSSL 3.0.0 final (GH-28205) (GH-28217)
7a6178a

ambv · 2021-09-08T17:03:22Z

Christian, Python is now tested with 3.0.0 final in 3.9, 3.10, and 3.11. Looks like we can close this!

Thank you for this big body of work ✨ 🍰 ✨

ambv · 2021-09-08T17:05:14Z

(I'll let you close this yourself when you determine that the two remaining open dependencies can be closed as well.)

Steve123p · 2022-04-11T19:27:16Z

Is this going to be officially closed?

kumaraditya303 · 2022-05-09T12:51:55Z

I am closing this as it is fixed, if required it can reopened.

tiran added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes labels Nov 16, 2019

tiran self-assigned this Nov 16, 2019

tiran added topic-SSL type-feature A feature request or enhancement labels Nov 16, 2019

methane closed this as completed Oct 17, 2020

tiran reopened this Oct 17, 2020

hroncok mannequin added the 3.10 only security fixes label Apr 7, 2021

tiran removed the 3.7 (EOL) end of life label Apr 9, 2021

ambv added 3.11 only security fixes and removed 3.8 only security fixes labels Sep 8, 2021

ezio-melotti transferred this issue from another repository Apr 10, 2022

berghaus mentioned this issue May 3, 2022

TLS/SSL connection has been closed (EOF) with TLSv1 OfflineIMAP/offlineimap3#118

Open

kumaraditya303 closed this as completed May 9, 2022

native-api mentioned this issue May 11, 2022

Segmentation fault when installing Python 3.6.15 on Ubuntu 22.04 pyenv/pyenv#2359

Closed

11 tasks

mhsmith mentioned this issue Aug 31, 2022

Update SQLite and OpenSSL chaquo/chaquopy#688

Closed

pquentin mentioned this issue May 21, 2023

Support both urllib3 v1 and v2 at the same time (fixes #688) kevin1024/vcrpy#699

Merged

Edward-Knight mentioned this issue Jun 19, 2023

Building with OpenSSL 3.0 for the GitHub Release binaries indygreg/python-build-standalone#175

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make Python compatible with OpenSSL 3.0.0 #83001

Make Python compatible with OpenSSL 3.0.0 #83001

tiran commented Nov 16, 2019

tiran commented Nov 16, 2019

tiran commented Nov 16, 2019

miss-islington commented Dec 7, 2019

miss-islington commented Dec 7, 2019

miss-islington commented Dec 7, 2019

iritkatriel commented Oct 16, 2020

tiran commented Oct 17, 2020

hroncok mannequin commented Apr 7, 2021

tiran commented Apr 9, 2021

tiran commented Apr 9, 2021

miss-islington commented Apr 9, 2021

miss-islington commented Apr 9, 2021

tiran commented Apr 23, 2021

tiran commented Apr 26, 2021

miss-islington commented Apr 26, 2021

miss-islington commented Apr 26, 2021

tiran commented May 2, 2021

miss-islington commented May 6, 2021

miss-islington commented May 20, 2021

tiran commented Jun 19, 2021

miss-islington commented Jun 19, 2021

ambv commented Jul 12, 2021

ambv commented Sep 7, 2021

miss-islington commented Sep 8, 2021

ambv commented Sep 8, 2021

ambv commented Sep 8, 2021

ambv commented Sep 8, 2021

Steve123p commented Apr 11, 2022

kumaraditya303 commented May 9, 2022

Make Python compatible with OpenSSL 3.0.0 #83001

Make Python compatible with OpenSSL 3.0.0 #83001

Comments

tiran commented Nov 16, 2019

tiran commented Nov 16, 2019

tiran commented Nov 16, 2019

miss-islington commented Dec 7, 2019

miss-islington commented Dec 7, 2019

miss-islington commented Dec 7, 2019

iritkatriel commented Oct 16, 2020

tiran commented Oct 17, 2020

hroncok mannequin commented Apr 7, 2021

tiran commented Apr 9, 2021

tiran commented Apr 9, 2021

miss-islington commented Apr 9, 2021

miss-islington commented Apr 9, 2021

tiran commented Apr 23, 2021

tiran commented Apr 26, 2021

miss-islington commented Apr 26, 2021

miss-islington commented Apr 26, 2021

tiran commented May 2, 2021

miss-islington commented May 6, 2021

miss-islington commented May 20, 2021

tiran commented Jun 19, 2021

miss-islington commented Jun 19, 2021

ambv commented Jul 12, 2021

ambv commented Sep 7, 2021

miss-islington commented Sep 8, 2021

ambv commented Sep 8, 2021

ambv commented Sep 8, 2021

ambv commented Sep 8, 2021

Steve123p commented Apr 11, 2022

kumaraditya303 commented May 9, 2022