Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update SQLite to 3.31.1 in Windows and macOS installer builds #82561

Closed
BigStone mannequin opened this issue Oct 5, 2019 · 30 comments
Closed

Update SQLite to 3.31.1 in Windows and macOS installer builds #82561

BigStone mannequin opened this issue Oct 5, 2019 · 30 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes build The build process and cross-build OS-mac OS-windows

Comments

@BigStone
Copy link
Mannequin

BigStone mannequin commented Oct 5, 2019

BPO 38380
Nosy @pfmoore, @ronaldoussoren, @tjguk, @benjaminp, @ned-deily, @zware, @zooba, @miss-islington, @erlend-aasland
PRs
  • bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 #18678
  • [3.8] bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678) #18740
  • [3.7] bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678) #18741
    		•
    Files
  • 0001-bpo-38380-Update-macOS-installer-to-use-SQLite-3.30..patch: bpo-38380: Update macOS installer to use SQLite 3.30.0
  • 0002-bpo-38380-Update-Windows-builds-to-use-SQLite-3.30.0.patch: bpo-38380: Update Windows builds to use SQLite 3.30.0
  • 0001-bpo-38380-Update-macOS-installer-to-use-SQLite-3.30.1.patch
  • 0002-bpo-38380-Update-Windows-builds-to-use-SQLite-3.30.1.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-03-04.02:45:36.925>
created_at = <Date 2019-10-05.22:12:20.397>
labels = ['OS-mac', '3.7', '3.8', '3.9', 'build', 'OS-windows']
title = 'Update SQLite to 3.31.1 in Windows and macOS installer builds'
updated_at = <Date 2020-03-04.02:45:36.924>
user = 'https://bugs.python.org/BigStone'

    bugs.python.org fields:

    activity = <Date 2020-03-04.02:45:36.924>
actor = 'benjamin.peterson'
assignee = 'none'
closed = True
closed_date = <Date 2020-03-04.02:45:36.925>
closer = 'benjamin.peterson'
components = ['Build', 'macOS', 'Windows']
creation = <Date 2019-10-05.22:12:20.397>
creator = 'Big Stone'
dependencies = []
files = ['48651', '48652', '48921', '48922']
hgrepos = []
issue_num = 38380
keywords = ['patch']
message_count = 30.0
messages = ['354023', '354025', '354280', '354281', '354282', '354284', '354300', '362803', '362923', '362928', '362929', '362930', '362933', '362935', '362938', '362963', '362970', '362973', '363144', '363156', '363158', '363160', '363161', '363162', '363166', '363178', '363179', '363180', '363182', '363323']
nosy_count = 10.0
nosy_names = ['paul.moore', 'ronaldoussoren', 'tim.golden', 'benjamin.peterson', 'ned.deily', 'zach.ware', 'steve.dower', 'Big Stone', 'miss-islington', 'erlendaasland']
pr_nums = ['18678', '18740', '18741']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue38380'
versions = ['Python 2.7', 'Python 3.7', 'Python 3.8', 'Python 3.9']
    @BigStone
    Copy link
    Mannequin Author

    BigStone mannequin commented Oct 5, 2019

    there is a security fix in sqlite-3.30

    https://nvd.nist.gov/vuln/detail/CVE-2019-16168#VulnChangeHistorySection

    https://www.sqlite.org/releaselog/3_30_0.html

    @ned-deily
    Copy link
    Member

    I verified it is exploitable via the sqlite3 module by adapting the test case from the SQLite ticket (https://www.sqlite.org/src/info/e4598ecbdd18bd82). But since it requires the exploiter to be able to specify raw SQL statements, it doesn't sound like it needs to be treated as a Python security issue. We should plan to to update to the latest SQLite but it doesn't need to be a release blocker.

    @ned-deily ned-deily added build The build process and cross-build OS-mac OS-windows 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes labels Oct 5, 2019
    @erlend-aasland
    Copy link
    Contributor

    I've prepared a PR for https://github.com/python/cpython-source-deps at https://github.com/erlend-aasland/cpython-source-deps/tree/upgrade-sqlite.

    Patches for Windows and macOS installer builds on 3.9 prepared at https://github.com/erlend-aasland/cpython/tree/fix-issue-38380.

    If it looks ok, I'll create PR's for CPython and cpython-source-deps. I guess this should be backported to 3.8, 3.7 and 2.7.

    @erlend-aasland
    Copy link
    Contributor

    bpo-38380: Update macOS installer to use SQLite 3.30.0
    erlend-aasland@aa7d7b1

    @erlend-aasland
    Copy link
    Contributor

    bpo-38380: Update Windows builds to use SQLite 3.30.0
    erlend-aasland@e25214e

    @erlend-aasland
    Copy link
    Contributor

    FYI: Compiled cpython 3.9 with sqlite-3.30 on macOS 10.14.6. Make test completes without errors.

    @erlend-aasland
    Copy link
    Contributor

    Update: Tested on macOS 10.14.6 with make test on 2.7.17rc1, 3.7.5rc1+, 3.8.0rc1+.

    @erlend-aasland
    Copy link
    Contributor

    I've updated these patches to sqlite3-3.30.1 (https://www.sqlite.org/releaselog/3_30_1.html).

    cpython-source-deps is updated (python/cpython-source-deps#17).

    CPython commits are updated and rebased onto current 3.9 master: https://github.com/erlend-aasland/cpython/tree/fix-issue-38380

    (Since Oct 2019, SQLite 3.31 have been released, but that would probably require opening a new issue.)

    @zooba
    Copy link
    Member

    zooba commented Feb 28, 2020

    We still need the tag added to the cpython-source-deps repo, and I still can't complete a clone right now (something is strange with SSL to GitHub on my (temporary) internet connection).

    Zach - can you tag it? python/cpython-source-deps#17

    @erlend-aasland
    Copy link
    Contributor

    Yes, we need the tag for the Windows build, so the PR currently fails the Windows checks. (Tagging must be done explicitly by the maintainers, IIRC.) I also forgot to add a NEWS entry, so I'll do another push to add those (and kick off the CI) when the tag arrives.

    @ned-deily
    Copy link
    Member

    Thanks for the PRs. If we're going to update now as we should, why not to 3.31.1 which is current?

    @erlend-aasland
    Copy link
    Contributor

    You're welcome. If you are ok with that, I'd be happy to prepare a PR for the source deps for sqlite3 v3.31.1, and update #62878 as soon as it is tagged.

    @ned-deily
    Copy link
    Member

    I would prefer to go to 3.31.1 at this point particularly given the track record of the SQLite project. It's been released for a month now. Any objections, Steve?

    @erlend-aasland
    Copy link
    Contributor

    I agree. I've updated the branches for source deps and cpython. I'll wait for Steve's approval before I open a new PR over at cpython-source-deps and update #62878.

    @zooba
    Copy link
    Member

    zooba commented Feb 28, 2020

    Isn't that what we ended up merging? (Goes to check). Ah, that was 3.30.1.

    Sure, go for it. We'll have RCs of everything before the next final releases go out, so provided someone double checks that it's all good before then I'm okay with it.

    @erlend-aasland
    Copy link
    Contributor

    Great, thanks folks! I've pushed an update to #62878.

    (BTW, is it kosher to force push to PRs like this?)

    @erlend-aasland erlend-aasland changed the title Update SQLite to 3.30 in Windows and macOS installer builds Update SQLite to 3.31.1 in Windows and macOS installer builds Feb 29, 2020
    @zooba
    Copy link
    Member

    zooba commented Feb 29, 2020

    (BTW, is it kosher to force push to PRs like this?)

    We always squash merge, so you don't need to do it to hide your history. Personally, I do it occasionally when the history is very long and nobody has started reviewing.

    Force pushing will reset any reviews, so you'll be forcing reviewers to start again instead of just looking at your updates. Some reviewers (e.g. me) will often defer a full rereview if we don't have a lot of time and someone just force pushed. So in general, avoid it.

    @erlend-aasland
    Copy link
    Contributor

    Understood; I’ll remember it for future PRs. Thanks for clarifying.

    @erlend-aasland
    Copy link
    Contributor

    Would you prefer that I close GH_18678 and split it up in separate RPs for macOS and Windows?

    @zooba
    Copy link
    Member

    zooba commented Mar 2, 2020

    No, that one is fine.

    I just need someone else to jump in and tag the commit in cpython-source-deps for me (because I can't clone it successfully on my current internet connection).

    @erlend-aasland
    Copy link
    Contributor

    Ah, but you're all set @steve.dower, someone at cpython-source-deps already pulled my tags: https://github.com/python/cpython-source-deps/tags :)

    But I guess that if I were to do the same maneouver again, it would probably be preferable if I made one PR for macOS and another for Windows, right?

    @zooba
    Copy link
    Member

    zooba commented Mar 2, 2020

    Nope, same PR and bug is totally fine.

    Putting the NEWS items in their platform-specific sections is how we normally do it (though to be fair, I had to go look it up).

    @erlend-aasland
    Copy link
    Contributor

    Ok, thanks for your patience and time. I'll fix the NEWS entries right away.

    @zooba
    Copy link
    Member

    zooba commented Mar 2, 2020

    I marked the PR to backport to 3.7 and 3.8.

    Up to Benjamin whether 2.7 gets it, but unless there's a specific and impactful CVE that's been fixed, I doubt it (the one linked at the start of this issue seems to require direct modification of the SQL statement, which would be a bug in itself if permitted, so I think it's outside of our threat model for CPython).

    @miss-islington
    Copy link
    Contributor

    New changeset 1382c32 by Erlend Egeberg Aasland in branch 'master':
    bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678)
    1382c32

    @miss-islington
    Copy link
    Contributor

    New changeset 7ad9982 by Miss Islington (bot) in branch '3.8':
    bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678)
    7ad9982

    @miss-islington
    Copy link
    Contributor

    New changeset 7ca251b by Miss Islington (bot) in branch '3.7':
    bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678)
    7ca251b

    @zooba
    Copy link
    Member

    zooba commented Mar 2, 2020

    Leaving open until someone (cough Benjamin) confirms whether this has to go into 2.7 or not.

    @erlend-aasland
    Copy link
    Contributor

    Cool. Should I remove Python-3.* from the Versions label, so it doesn't show up in searches for 3.* issues?

    @benjaminp
    Copy link
    Contributor

    Traditionally, we've taken these dependency updates in 2.7, but there doesn't seem to be a compelling reason to take this one.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes build The build process and cross-build OS-mac OS-windows
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    5 participants
    @benjaminp @zooba @ned-deily @erlend-aasland @miss-islington