Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

multissl doesn't support tarballs in /source/old/ #84344

Closed
vstinner opened this issue Apr 3, 2020 · 9 comments
Closed

multissl doesn't support tarballs in /source/old/ #84344

vstinner opened this issue Apr 3, 2020 · 9 comments
Assignees
@tiran
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@vstinner
Copy link
Member

vstinner commented Apr 3, 2020

BPO 40163
Nosy @vstinner, @larryhastings, @tiran, @benjaminp, @miss-islington
PRs
  • bpo-40163: Fix multissltest download of old OpenSSL (GH-19329) #19329
  • [3.8] bpo-40163: Fix multissltest download of old OpenSSL (GH-19329) #20122
  • [3.7] bpo-40163: Fix multissltest download of old OpenSSL (GH-19329) #20123
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tiran'
closed_at = <Date 2020-05-15.21:05:41.744>
created_at = <Date 2020-04-03.00:48:58.545>
labels = ['3.7', '3.8', 'type-bug', 'tests', '3.9']
title = "multissl doesn't support tarballs in /source/old/"
updated_at = <Date 2020-05-16.09:44:58.185>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2020-05-16.09:44:58.185>
actor = 'vstinner'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2020-05-15.21:05:41.744>
closer = 'christian.heimes'
components = ['Demos and Tools', 'Tests']
creation = <Date 2020-04-03.00:48:58.545>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 40163
keywords = ['patch']
message_count = 9.0
messages = ['365654', '365656', '365674', '365675', '365694', '368968', '368973', '368974', '369028']
nosy_count = 5.0
nosy_names = ['vstinner', 'larry', 'christian.heimes', 'benjamin.peterson', 'miss-islington']
pr_nums = ['19329', '20122', '20123']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40163'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']
    @vstinner
    Copy link
    Member Author

    vstinner commented Apr 3, 2020

    Tools/ssl/multissltests.py expects to find OpenSSL tarballs in:
    https://www.openssl.org/source/

    Like: https://www.openssl.org/source/openssl-1.1.1f.tar.gz

    Problem: OpenSSL moves old versions to https://www.openssl.org/source/old/

    If Tools/ssl/multissltests.py fails to download a tarball (HTTP error 404), it should try to get it from /source/old/.

    It would prevent us to have to upgrade OpenSSL version immediately in all Python branches of all CIs (Azure Pipelines and Travis CI) as soon as OpenSSL decides to move a tarball. This move is not under our control.

    Upgrading OpenSSL is a good practice. Breaking our CI is not :-)

    @vstinner vstinner added 3.9 only security fixes labels Apr 3, 2020
    @vstinner
    Copy link
    Member Author

    vstinner commented Apr 3, 2020

    When OpenSSL moves a tarball, all our pre-commit CIs are broken and suddenly, all PRs can no longer be merged. We have first write PRs to update the configuration of our CI to use the newer OpenSSL version, merge these PRs, and then *all* pending PRs must be rebased on top of these merged PRS to retrieve the newer CI configuration.

    There are currently 1085 pending PRs at https://github.com/python/cpython/pulls Well, for most of them, the CI already passed so we can merge them. But if a reviewer requires changes, the CI will re-run and then fail :-(

    Moreover, fixing multissltests.py doesn't help neither, since again, PRs should be rebased to retrieve multissltests.py changes.

    I hope that I'm wrong and the situation is not so bad.

    --

    Another solution would be to enhance our workflow to always rebase PRs on the development branch. Something like what https://mergify.io/ does.

    I'm not sure what is the configuration of Azure Pipelines, GitHub actions and Travis CI. Would it be possible to make them rebase the PRs before running tests.

    @tiran
    Copy link
    Member

    tiran commented Apr 3, 2020

    Ah crap :/ That's annoying.

    This breaks all CI of all our active branches and all open PRs. I'll fix the issue and talk to OpenSSL upstream.

    @tiran tiran added 3.7 (EOL) end of life 3.8 only security fixes labels Apr 3, 2020
    @tiran tiran self-assigned this Apr 3, 2020
    @tiran tiran added 3.7 (EOL) end of life 3.8 only security fixes labels Apr 3, 2020
    @tiran tiran self-assigned this Apr 3, 2020
    @tiran
    Copy link
    Member

    tiran commented Apr 3, 2020

    Benjamin, Larry,
    The problem affects testing of security-only branches and 2.7.

    @tiran tiran added tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error labels Apr 3, 2020
    @vstinner
    Copy link
    Member Author

    vstinner commented Apr 3, 2020

    talk to OpenSSL upstream.

    Do you mean continue to provide old versions in /source/ directory as well? Maybe they move tarballs to /source/old/ on purpose, to force users to use the latest versions which get fixes for new vulnerabilities?

    @tiran
    Copy link
    Member

    tiran commented May 15, 2020

    New changeset 938717f by Christian Heimes in branch 'master':
    bpo-40163: Fix multissltest download of old OpenSSL (GH-19329)
    938717f

    @miss-islington
    Copy link
    Contributor

    New changeset 7a89f9b by Miss Islington (bot) in branch '3.7':
    bpo-40163: Fix multissltest download of old OpenSSL (GH-19329)
    7a89f9b

    @miss-islington
    Copy link
    Contributor

    New changeset fcea080 by Miss Islington (bot) in branch '3.8':
    bpo-40163: Fix multissltest download of old OpenSSL (GH-19329)
    fcea080

    @tiran tiran closed this as completed May 15, 2020
    @tiran tiran closed this as completed May 15, 2020
    @vstinner
    Copy link
    Member Author

    Thanks for the fix!

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @tiran tiran

    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @tiran @miss-islington