This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: [sqlite3] lazy creation of result rows
Type: enhancement Stage: resolved
Components: Extension Modules Versions:
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: berker.peksag, erlendaasland, pablogsal, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2021-08-22 09:06 by erlendaasland, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27884 merged erlendaasland, 2021-08-22 09:08
Messages (3)
msg400062 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-08-22 09:06
Currently, we build the first result row in the _pysqlite_query_execute() loop if sqlite3_step() returned SQLITE_ROW. When the user asks for a row (for example, using sqlite3.Cursor.fetchone()), this pre-built row is returned, and the next row is prepared.

Suggesting to lazily build result rows instead.

- no result tuples are built unless sqlite3.Cursor.fetch*() is called
- no need to keep the next result row (tuple) in pysqlite_Cursor; rows are built on demand
- pysqlite_cursor_iternext() is vastly simplified (50% less lines of code)
- the main loop in _pysqlite_query_execute() is further simplified

- code churn

git diff main --shortstat: 2 files changed, 29 insertions(+), 58 deletions(-)
msg400065 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-08-22 09:26
Historical note:
The current behaviour was introduced in pysqlite v2.0.2 (May 21 2005), in order to be able to finalise statements and thus close cursors as soon as possible. However, that effect was cancelled just after a couple of months, by the addition of the LRU cache and the ability to reuse prepared statements.
msg400252 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-25 10:28
New changeset 3df0fc89bc2714f5ef03e36a926bc795dcd5e05a by Erlend Egeberg Aasland in branch 'main':
bpo-44976: Lazy creation of sqlite3 result rows (GH-27884)
Date User Action Args
2022-04-11 14:59:49adminsetgithub: 89139
2021-08-25 10:29:18pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-08-25 10:28:55pablogsalsetnosy: + pablogsal
messages: + msg400252
2021-08-22 09:26:33erlendaaslandsetmessages: + msg400065
2021-08-22 09:08:58erlendaaslandsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26339
2021-08-22 09:06:30erlendaaslandcreate