classification
Title: multissl doesn't support tarballs in /source/old/
Type: behavior Stage: resolved
Components: Demos and Tools, Tests Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: benjamin.peterson, christian.heimes, larry, miss-islington, vstinner
Priority: normal Keywords: patch

Created on 2020-04-03 00:48 by vstinner, last changed 2020-05-16 09:44 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19329 merged christian.heimes, 2020-04-03 06:36
PR 20122 merged miss-islington, 2020-05-15 20:33
PR 20123 merged miss-islington, 2020-05-15 20:33
Messages (9)
msg365654 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-03 00:48
Tools/ssl/multissltests.py expects to find OpenSSL tarballs in:
https://www.openssl.org/source/

Like: https://www.openssl.org/source/openssl-1.1.1f.tar.gz

Problem: OpenSSL moves old versions to https://www.openssl.org/source/old/

If Tools/ssl/multissltests.py fails to download a tarball (HTTP error 404), it should try to get it from /source/old/.

It would prevent us to have to upgrade OpenSSL version immediately in all Python branches of all CIs (Azure Pipelines and Travis CI) as soon as OpenSSL decides to move a tarball. This move is not under our control.

Upgrading OpenSSL is a good practice. Breaking our CI is not :-)
msg365656 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-03 00:55
When OpenSSL moves a tarball, all our pre-commit CIs are broken and suddenly, all PRs can no longer be merged. We have first write PRs to update the configuration of our CI to use the newer OpenSSL version, merge these PRs, and then *all* pending PRs must be rebased on top of these merged PRS to retrieve the newer CI configuration.

There are currently 1085 pending PRs at https://github.com/python/cpython/pulls Well, for most of them, the CI already passed so we can merge them. But if a reviewer requires changes, the CI will re-run and then fail :-(

Moreover, fixing multissltests.py doesn't help neither, since again, PRs should be rebased to retrieve multissltests.py changes.

I hope that I'm wrong and the situation is not so bad.

--

Another solution would be to enhance our workflow to always rebase PRs on the development branch. Something like what https://mergify.io/ does.

I'm not sure what is the configuration of Azure Pipelines, GitHub actions and Travis CI. Would it be possible to make them rebase the PRs before running tests.
msg365674 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-04-03 06:06
Ah crap :/ That's annoying.

This breaks all CI of all our active branches and all open PRs. I'll fix the issue and talk to OpenSSL upstream.
msg365675 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-04-03 06:38
Benjamin, Larry,
The problem affects testing of security-only branches and 2.7.
msg365694 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-03 12:13
> talk to OpenSSL upstream.

Do you mean continue to provide old versions in /source/ directory as well? Maybe they move tarballs to /source/old/ on purpose, to force users to use the latest versions which get fixes for new vulnerabilities?
msg368968 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-05-15 20:32
New changeset 938717fd04c53c717895a756d5910e8c8813706c by Christian Heimes in branch 'master':
bpo-40163: Fix multissltest download of old OpenSSL (GH-19329)
https://github.com/python/cpython/commit/938717fd04c53c717895a756d5910e8c8813706c
msg368973 - (view) Author: miss-islington (miss-islington) Date: 2020-05-15 20:55
New changeset 7a89f9b4e2c05a6abdf59e8a96a1fc80a47a1144 by Miss Islington (bot) in branch '3.7':
bpo-40163: Fix multissltest download of old OpenSSL (GH-19329)
https://github.com/python/cpython/commit/7a89f9b4e2c05a6abdf59e8a96a1fc80a47a1144
msg368974 - (view) Author: miss-islington (miss-islington) Date: 2020-05-15 20:55
New changeset fcea08059f46d2d9582bb7ce5b2e905b20b86e8e by Miss Islington (bot) in branch '3.8':
bpo-40163: Fix multissltest download of old OpenSSL (GH-19329)
https://github.com/python/cpython/commit/fcea08059f46d2d9582bb7ce5b2e905b20b86e8e
msg369028 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-16 09:44
Thanks for the fix!
History
Date User Action Args
2020-05-16 09:44:58vstinnersetmessages: + msg369028
2020-05-15 21:05:41christian.heimessetstatus: open -> closed
resolution: fixed
stage: backport needed -> resolved
2020-05-15 20:55:48miss-islingtonsetmessages: + msg368974
2020-05-15 20:55:01miss-islingtonsetmessages: + msg368973
2020-05-15 20:34:22christian.heimessetstage: patch review -> backport needed
versions: - Python 2.7, Python 3.5, Python 3.6
2020-05-15 20:33:55miss-islingtonsetpull_requests: + pull_request19427
2020-05-15 20:33:46miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request19426
2020-05-15 20:32:31christian.heimessetmessages: + msg368968
2020-04-03 12:13:34vstinnersetmessages: + msg365694
2020-04-03 06:38:22christian.heimessetnosy: + larry, benjamin.peterson
messages: + msg365675

components: + Tests
type: behavior
2020-04-03 06:36:25christian.heimessetkeywords: + patch
stage: patch review
pull_requests: + pull_request18694
2020-04-03 06:06:30christian.heimessetassignee: christian.heimes
messages: + msg365674
versions: + Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8
2020-04-03 00:55:10vstinnersetmessages: + msg365656
2020-04-03 00:48:58vstinnercreate