This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: [sqlite3] add support for changing connection limits
Type: enhancement Stage: resolved
Components: Extension Modules Versions:
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: erlendaasland Nosy List: berker.peksag, christian.heimes, erlendaasland, pablogsal, serhiy.storchaka, steve.dower
Priority: low Keywords: patch

Created on 2021-09-19 22:35 by erlendaasland, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28463 merged erlendaasland, 2021-09-19 23:29
PR 28790 closed erlendaasland, 2021-10-07 12:47
PR 29356 merged erlendaasland, 2021-11-02 00:25
Messages (5)
msg402176 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-09-19 22:35
I propose to add wrappers for the SQLite sqlite3_limit() C API. Using this API, it is possible to query and set limits on a connection basis. This will make it easier (and faster) to test various corner cases in the test suite without relying on

Quoting from the SQLite sqlite3_limit() docs:

  Run-time limits are intended for use in applications that manage both their
  own internal database and also databases that are controlled by untrusted
  external sources. An example application might be a web browser that has its
  own databases for storing history and separate databases controlled by
  JavaScript applications downloaded off the Internet. The internal databases
  can be given the large, default limits. Databases managed by external
  sources can be given much smaller limits designed to prevent a denial of
  service attack.

See also:

Limit categories (C&P from SQLite docs)

The maximum size of any string or BLOB or table row, in bytes.

The maximum length of an SQL statement, in bytes.

The maximum number of columns in a table definition or in the result set of a SELECT or the maximum number of columns in an index or in an ORDER BY or GROUP BY clause.

The maximum depth of the parse tree on any expression.

The maximum number of terms in a compound SELECT statement.

The maximum number of instructions in a virtual machine program used to implement an SQL statement. If sqlite3_prepare_v2() or the equivalent tries to allocate space for more than this many opcodes in a single prepared statement, an SQLITE_NOMEM error is returned.

The maximum number of arguments on a function.

The maximum number of attached databases.

The maximum length of the pattern argument to the LIKE or GLOB operators.

The maximum index number of any parameter in an SQL statement.

The maximum depth of recursion for triggers.

The maximum number of auxiliary worker threads that a single prepared statement may start.
msg402186 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-09-19 23:22
Christian, how about adding an audit event for something like sqlite3.Connection.setlimit()? My initial thought is: yes.
msg405474 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-11-01 22:51
New changeset b6b38a82267ff70d2abaf2a8371327268887c97d by Erlend Egeberg Aasland in branch 'main':
bpo-45243: Add support for setting/getting `sqlite3` connection limits (GH-28463)
msg405566 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-11-02 22:28
Steve, do you think it is worth it adding an audit hook for setting connection limits?

Most of the limits are harmless, but limits that control recursion are more interesting.


    Maximum Depth Of An Expression Tree

    SQLite parses expressions into a tree for processing. During code
    generation, SQLite walks this tree recursively. The depth of expression
    trees is therefore limited in order to avoid using too much stack space.
    [...] If the value is 0, then no limit is enforced.


    Maximum Depth Of Trigger Recursion

    SQLite limits the depth of recursion of triggers in order to prevent a
    statement involving recursive triggers from using an unbounded amount of

Note also, how the SQLite docs talk about SQLITE_LIMIT_LENGTH:

    Maximum length of a string or BLOB

    [...] In security-sensitive applications it is best not to try to increase
    the maximum string and blob length. In fact, you might do well to lower
    the maximum string and blob length to something more in the range of a few
    million if that is possible.
msg405811 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-11-05 17:19
New changeset 3d42cd9461e60c7427f3793f640cd975fbd99289 by Erlend Egeberg Aasland in branch 'main':
bpo-45243: Use connection limits to simplify `sqlite3` tests (GH-29356)
Date User Action Args
2022-04-11 14:59:50adminsetgithub: 89406
2021-11-13 21:17:03erlendaaslandsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-11-05 17:19:50serhiy.storchakasetmessages: + msg405811
2021-11-02 22:28:00erlendaaslandsetnosy: + steve.dower
messages: + msg405566
2021-11-02 00:25:14erlendaaslandsetpull_requests: + pull_request27616
2021-11-01 22:51:02pablogsalsetnosy: + pablogsal
messages: + msg405474
2021-10-07 12:47:32erlendaaslandsetpull_requests: + pull_request27123
2021-09-19 23:29:31erlendaaslandsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26864
2021-09-19 23:22:04erlendaaslandsetnosy: + christian.heimes
messages: + msg402186
2021-09-19 22:35:54erlendaaslandcreate