classification
Title: Update SQLite to 3.28 in Windows and macOS installer builds
Type: Stage: resolved
Components: Build, macOS, Windows Versions: Python 3.8, Python 3.7, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Big Stone, Ma Lin, Mariatta, Scott Stevens, berker.peksag, ceronman, ghaering, miss-islington, ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, zach.ware
Priority: high Keywords: patch

Created on 2018-11-30 12:14 by vstinner, last changed 2019-09-30 14:52 by steve.dower. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 14179 merged Ma Lin, 2019-06-18 00:52
PR 14180 merged Ma Lin, 2019-06-18 01:01
PR 14182 merged Ma Lin, 2019-06-18 01:56
PR 14183 merged Ma Lin, 2019-06-18 02:04
PR 14195 merged miss-islington, 2019-06-18 09:25
PR 14196 merged miss-islington, 2019-06-18 09:25
PR 14352 merged miss-islington, 2019-06-24 23:22
PR 14353 merged miss-islington, 2019-06-24 23:22
PR 14354 closed steve.dower, 2019-06-24 23:26
Messages (25)
msg330779 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-30 12:14
Windows and macOS installers require SQLite, but they require different versions! Windows uses 3.21 or 3.14, but macOS uses 3.22.

I'm talking about the following line in PCbuild\get_externals.bat:

   set libraries=%libraries% sqlite-3.21.0.0


* 3.6, 3.7 and master branches:

SQLite[Windows]: 3.21.0.0
SQLite[macOS]: 3.22.0

* 2.7 branch:

SQLite[Windows]: 3.14.2.0
SQLite[macOS]: 3.22.0


Note: I wrote a script to get external dependencies:

https://github.com/vstinner/misc/blob/master/cpython/external_versions.py
msg330783 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-30 12:32
First the sqlite branch should be updated in:
https://github.com/python/cpython-source-deps

Then a new tag should be created in this repository.

I tried to update cpython-source-deps, but "git push" never completed. I will retry next week ;-)
msg331947 - (view) Author: Scott Stevens (Scott Stevens) Date: 2018-12-17 07:07
With the discovery of the SQLite "Magellan" bug, could the version be  upgraded to 3.26 for all Python versions? As far as I know, the security case is restricted to where the user is allowing aribitrary SQL execution without arbitrary Python execution, but in that case I do believe remote code execution is possible.

https://blade.tencent.com/magellan/index_en.html
msg332014 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-12-17 15:52
SQLite updates and changes for us almost always get stuck on someone being willing to verify that nothing has broken (and stand by their analysis).

Without an active expert (I just nosied ghaering in case they are around), I'm not confident to make this change in any version prior to 3.8.
msg332261 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-12-20 21:22
We should look at doing this for the next set of maintenance updates.  This doesn't appear to be critical enough to delay current releases unless someone can show how this exploit could be used in a typical Python application.
msg340978 - (view) Author: Big Stone (Big Stone) Date: 2019-04-27 10:37
sqlite-3.28.0 now available, with extended window functio support: EXCLUDE clause, GROUPS frame types, window chaining, and support for "<expr> PRECEDING" and "<expr> FOLLOWING" boundaries in RANGE frames.
msg343456 - (view) Author: Big Stone (Big Stone) Date: 2019-05-25 07:22
any hope to have a SQLite refresh in Python-3.8.0b1 for Windows/Mac ?

 It's generally the ideal / less annoying moment to do so.
msg343531 - (view) Author: Ma Lin (Ma Lin) * Date: 2019-05-26 04:26
@Mariatta Wijaya, would you update SQLite?

I want to do it myself, by following your patch in issue28791.
But I find I have to commit SQLite's source code to https://github.com/python/cpython-source-deps, so I think this should be done by a core developer.
msg345708 - (view) Author: Big Stone (Big Stone) Date: 2019-06-15 17:47
any hope for beta2 ?
msg345865 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-06-17 15:23
Has anyone tried building with the newer SQLite and confirmed that the tests and any scenarios they use still work?

We don't currently have a SQLite expert to do these kinds of things.
msg345890 - (view) Author: Big Stone (Big Stone) Date: 2019-06-17 19:25
on my test cases, sqlite_bro and baresql, db.py , it works without problems.
msg345901 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-06-17 20:30
I just pushed sources with tag sqlite-3.28.0.0, so now anybody should be able to submit a CPython PR for it.
msg345939 - (view) Author: Ma Lin (Ma Lin) * Date: 2019-06-18 01:29
PR 14179 is for Windows build
PR 14180 is for Mac OS X build

Both update to Sqlite 3.28.0
msg345941 - (view) Author: Ma Lin (Ma Lin) * Date: 2019-06-18 02:07
2.7 branch:

PR 14182 is for Windows build
PR 14183 is for Mac OS X build
msg345970 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-18 09:25
New changeset d8f336fdc10decdd82d3bc81a63aea8be149c0c8 by Ned Deily (animalize) in branch 'master':
bpo-35360: Update macOS installer to use SQLite 3.28.0 (GH-14180)
https://github.com/python/cpython/commit/d8f336fdc10decdd82d3bc81a63aea8be149c0c8
msg345974 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-18 10:00
New changeset 373dace8d7ee5107cb38cd2e4fa5fd67dcad42dd by Ned Deily (animalize) in branch '2.7':
[2.7] bpo-35360: Update macOS installer to use SQLite 3.28.0 (GH-14183)
https://github.com/python/cpython/commit/373dace8d7ee5107cb38cd2e4fa5fd67dcad42dd
msg345977 - (view) Author: miss-islington (miss-islington) Date: 2019-06-18 10:15
New changeset a7072ff56e216683f949b353255cce099f5cdb24 by Miss Islington (bot) in branch '3.8':
bpo-35360: Update macOS installer to use SQLite 3.28.0 (GH-14180)
https://github.com/python/cpython/commit/a7072ff56e216683f949b353255cce099f5cdb24
msg345979 - (view) Author: miss-islington (miss-islington) Date: 2019-06-18 10:24
New changeset 624c9a27b10df4355eb2caf04654c730dfa412dd by Miss Islington (bot) in branch '3.7':
bpo-35360: Update macOS installer to use SQLite 3.28.0 (GH-14180)
https://github.com/python/cpython/commit/624c9a27b10df4355eb2caf04654c730dfa412dd
msg346448 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-06-24 23:22
New changeset 7fd2ba354ec2304743ffd9ba620e07d113532264 by Steve Dower (animalize) in branch 'master':
bpo-35360: Update Windows builds to use SQLite 3.28.0 (GH-14179)
https://github.com/python/cpython/commit/7fd2ba354ec2304743ffd9ba620e07d113532264
msg346449 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-06-24 23:27
New changeset 0fc14b373398767b4323a7bc2c2310b6218c5b74 by Steve Dower (animalize) in branch '2.7':
bpo-35360: Update Windows builds to use SQLite 3.28.0 (GH-14182)
https://github.com/python/cpython/commit/0fc14b373398767b4323a7bc2c2310b6218c5b74
msg346453 - (view) Author: miss-islington (miss-islington) Date: 2019-06-24 23:43
New changeset dad8f79cfd83b824decb15eac4c0d5723edd4bf1 by Miss Islington (bot) in branch '3.8':
bpo-35360: Update Windows builds to use SQLite 3.28.0 (GH-14179)
https://github.com/python/cpython/commit/dad8f79cfd83b824decb15eac4c0d5723edd4bf1
msg346455 - (view) Author: miss-islington (miss-islington) Date: 2019-06-24 23:44
New changeset 14c179f97dbb7047cb30e4a610b5d5b531a4f6f7 by Miss Islington (bot) in branch '3.7':
bpo-35360: Update Windows builds to use SQLite 3.28.0 (GH-14179)
https://github.com/python/cpython/commit/14c179f97dbb7047cb30e4a610b5d5b531a4f6f7
msg347161 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-07-02 22:34
New changeset c58fc3af75b54203b26008b6942709bb07d00fc6 by Ned Deily (Miss Islington (bot)) in branch '3.7':
bpo-35360: Update Windows builds to use SQLite 3.28.0 (GH-14179)
https://github.com/python/cpython/commit/c58fc3af75b54203b26008b6942709bb07d00fc6
msg353441 - (view) Author: Big Stone (Big Stone) Date: 2019-09-28 08:23
there will be a security fix in sqlite-3.30 around October 10th.

https://nvd.nist.gov/vuln/detail/CVE-2019-16168#VulnChangeHistorySection

https://www.sqlite.org/draft/releaselog/3_30_0.html
msg353583 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-09-30 14:52
Please file a new issue
History
Date User Action Args
2019-09-30 14:52:45steve.dowersetmessages: + msg353583
2019-09-28 08:23:10Big Stonesetmessages: + msg353441
2019-07-02 22:34:03ned.deilysetmessages: + msg347161
2019-06-24 23:44:30steve.dowersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-06-24 23:44:10miss-islingtonsetmessages: + msg346455
2019-06-24 23:43:34miss-islingtonsetmessages: + msg346453
2019-06-24 23:27:06steve.dowersetmessages: + msg346449
2019-06-24 23:26:23steve.dowersetpull_requests: + pull_request14171
2019-06-24 23:22:35miss-islingtonsetpull_requests: + pull_request14170
2019-06-24 23:22:27miss-islingtonsetpull_requests: + pull_request14169
2019-06-24 23:22:17steve.dowersetmessages: + msg346448
2019-06-18 10:24:25miss-islingtonsetmessages: + msg345979
2019-06-18 10:15:59miss-islingtonsetnosy: + miss-islington
messages: + msg345977
2019-06-18 10:00:02ned.deilysetmessages: + msg345974
2019-06-18 09:25:53miss-islingtonsetpull_requests: + pull_request14033
2019-06-18 09:25:46miss-islingtonsetpull_requests: + pull_request14032
2019-06-18 09:25:26ned.deilysetmessages: + msg345970
2019-06-18 08:40:05vstinnersetnosy: - vstinner
2019-06-18 02:07:16Ma Linsetmessages: + msg345941
2019-06-18 02:04:59Ma Linsetpull_requests: + pull_request14020
2019-06-18 01:56:26Ma Linsetpull_requests: + pull_request14019
2019-06-18 01:29:18Ma Linsetmessages: + msg345939
title: Update SQLite to 3.26 in Windows and macOS installer builds -> Update SQLite to 3.28 in Windows and macOS installer builds
2019-06-18 01:01:15Ma Linsetpull_requests: + pull_request14018
2019-06-18 00:52:21Ma Linsetkeywords: + patch
stage: patch review
pull_requests: + pull_request14017
2019-06-17 21:00:10xtreaksetnosy: + berker.peksag
2019-06-17 20:30:57steve.dowersetmessages: + msg345901
2019-06-17 19:25:21Big Stonesetmessages: + msg345890
2019-06-17 15:23:21steve.dowersetmessages: + msg345865
2019-06-15 17:47:34Big Stonesetmessages: + msg345708
2019-05-26 04:26:04Ma Linsetnosy: + Mariatta, Ma Lin
messages: + msg343531
2019-05-25 07:22:27Big Stonesetmessages: + msg343456
2019-04-27 10:37:48Big Stonesetnosy: + Big Stone
messages: + msg340978
2019-02-27 19:47:08ceronmansetnosy: + ceronman
2018-12-20 21:22:12ned.deilysetpriority: normal -> high


components: + macOS
title: [Windows] Update SQLite dependency -> Update SQLite to 3.26 in Windows and macOS installer builds
nosy: + ned.deily, ronaldoussoren
versions: - Python 3.6
messages: + msg332261
2018-12-17 15:52:56steve.dowersetnosy: + ghaering
messages: + msg332014
2018-12-17 07:07:25Scott Stevenssetnosy: + Scott Stevens
messages: + msg331947
2018-11-30 12:32:04vstinnersetmessages: + msg330783
2018-11-30 12:14:37vstinnersetnosy: + paul.moore, tim.golden, zach.ware, steve.dower
components: + Windows
2018-11-30 12:14:30vstinnercreate