there is a security fix in sqlite-3.30

https://nvd.nist.gov/vuln/detail/CVE-2019-16168#VulnChangeHistorySection

https://www.sqlite.org/releaselog/3_30_0.html

I verified it is exploitable via the sqlite3 module by adapting the test case from the SQLite ticket (https://www.sqlite.org/src/info/e4598ecbdd18bd82).  But since it requires the exploiter to be able to specify raw SQL statements, it doesn't sound like it needs to be treated as a Python security issue.  We should plan to to update to the latest SQLite but it doesn't need to be a release blocker.

I've prepared a PR for https://github.com/python/cpython-source-deps at https://github.com/erlend-aasland/cpython-source-deps/tree/upgrade-sqlite. 

Patches for Windows and macOS installer builds on 3.9 prepared at https://github.com/erlend-aasland/cpython/tree/fix-issue-38380.

If it looks ok, I'll create PR's for CPython and cpython-source-deps. I guess this should be backported to 3.8, 3.7 and 2.7.

bpo-38380: Update macOS installer to use SQLite 3.30.0
https://github.com/erlend-aasland/cpython/commit/aa7d7b1a3bed9a6a73f0611d0542a3442e85b0b6

bpo-38380: Update Windows builds to use SQLite 3.30.0
https://github.com/erlend-aasland/cpython/commit/e25214e6fa7a64353d9c3e16b139c41f5d62eb31

FYI: Compiled cpython 3.9 with sqlite-3.30 on macOS 10.14.6. Make test completes without errors.

Update: Tested on macOS 10.14.6 with make test on 2.7.17rc1, 3.7.5rc1+, 3.8.0rc1+.

I've updated these patches to sqlite3-3.30.1 (https://www.sqlite.org/releaselog/3_30_1.html).

cpython-source-deps is updated (https://github.com/python/cpython-source-deps/pull/17).

CPython commits are updated and rebased onto current 3.9 master: https://github.com/erlend-aasland/cpython/tree/fix-issue-38380


(Since Oct 2019, SQLite 3.31 have been released, but that would probably require opening a new issue.)

We still need the tag added to the cpython-source-deps repo, and I still can't complete a clone right now (something is strange with SSL to GitHub on my (temporary) internet connection).

Zach - can you tag it? https://github.com/python/cpython-source-deps/pull/17

Yes, we need the tag for the Windows build, so the PR currently fails the Windows checks. (Tagging must be done explicitly by the maintainers, IIRC.) I also forgot to add a NEWS entry, so I'll do another push to add those (and kick off the CI) when the tag arrives.

Thanks for the PRs.  If we're going to update now as we should, why not to 3.31.1 which is current?

You're welcome. If you are ok with that, I'd be happy to prepare a PR for the source deps for sqlite3 v3.31.1, and update GH-18678 as soon as it is tagged.

I would prefer to go to 3.31.1 at this point particularly given the track record of the SQLite project. It's been released for a month now.  Any objections, Steve?

I agree. I've updated the branches for source deps and cpython. I'll wait for Steve's approval before I open a new PR over at cpython-source-deps and update GH-18678.

Isn't that what we ended up merging? (Goes to check). Ah, that was 3.30.1.

Sure, go for it. We'll have RCs of everything before the next final releases go out, so provided someone double checks that it's all good before then I'm okay with it.

Great, thanks folks! I've pushed an update to GH-18678.

(BTW, is it kosher to force push to PRs like this?)

> (BTW, is it kosher to force push to PRs like this?)

We always squash merge, so you don't need to do it to hide your history. Personally, I do it occasionally when the history is very long and nobody has started reviewing.

Force pushing will reset any reviews, so you'll be forcing reviewers to start again instead of just looking at your updates. Some reviewers (e.g. me) will often defer a full rereview if we don't have a lot of time and someone just force pushed. So in general, avoid it.

Understood; I’ll remember it for future PRs. Thanks for clarifying.

Would you prefer that I close GH_18678 and split it up in separate RPs for macOS and Windows?

No, that one is fine.

I just need someone else to jump in and tag the commit in cpython-source-deps for me (because I can't clone it successfully on my current internet connection).

Ah, but you're all set @steve.dower, someone at cpython-source-deps already pulled my tags: https://github.com/python/cpython-source-deps/tags :)

But I guess that if I were to do the same maneouver again, it would probably be preferable if I made one PR for macOS and another for Windows, right?

Nope, same PR and bug is totally fine.

Putting the NEWS items in their platform-specific sections is how we normally do it (though to be fair, I had to go look it up).

Ok, thanks for your patience and time. I'll fix the NEWS entries right away.

I marked the PR to backport to 3.7 and 3.8.

Up to Benjamin whether 2.7 gets it, but unless there's a specific and impactful CVE that's been fixed, I doubt it (the one linked at the start of this issue seems to require direct modification of the SQL statement, which would be a bug in itself if permitted, so I think it's outside of our threat model for CPython).

New changeset 1382c3289bcfd34ac6811fdf9aa5bc09ca8c320e by Erlend Egeberg Aasland in branch 'master':
bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678)
https://github.com/python/cpython/commit/1382c3289bcfd34ac6811fdf9aa5bc09ca8c320e

New changeset 7ad99821d8ae75222c50e69194a39f535bb058f5 by Miss Islington (bot) in branch '3.8':
bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678)
https://github.com/python/cpython/commit/7ad99821d8ae75222c50e69194a39f535bb058f5

New changeset 7ca251bd85f1182b9734579975c17fbd0488e2a4 by Miss Islington (bot) in branch '3.7':
bpo-38380: Update macOS & Windows builds to SQLite v3.31.1 (GH-18678)
https://github.com/python/cpython/commit/7ca251bd85f1182b9734579975c17fbd0488e2a4

Leaving open until someone (*cough* Benjamin) confirms whether this has to go into 2.7 or not.

Cool. Should I remove Python-3.* from the Versions label, so it doesn't show up in searches for 3.* issues?

Traditionally, we've taken these dependency updates in 2.7, but there doesn't seem to be a compelling reason to take this one.