This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [sqlite3] expand bound values in traced statements when possible
Type: enhancement Stage: patch review
Components: Extension Modules Versions: Python 3.11
process
Status: open Resolution: fixed
Dependencies: Superseder:
Assigned To: erlendaasland Nosy List: JelleZijlstra, erlendaasland, miss-islington
Priority: low Keywords: patch

Created on 2021-09-08 14:05 by erlendaasland, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 28240 merged erlendaasland, 2021-09-08 14:19
PR 31783 closed erlendaasland, 2022-03-09 12:12
PR 31788 merged erlendaasland, 2022-03-09 17:12
Messages (6)
msg401383 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-09-08 14:05
For SQLite 3.14.0 and newer, we're using the v2 trace API. This means that the trace callback receives a pointer to the sqlite3_stmt object. We can use the sqlite3_stmt pointer to retrieve expanded SQL string.

The following statement...:
cur.executemany("insert into t values(?)", ((v,) for v in range(3)))

...will produce the following traces:
  insert into t values(0)
  insert into t values(1)
  insert into t values(2)

...instead of:
  insert into t values(?)
  insert into t values(?)
  insert into t values(?)
msg414784 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2022-03-09 02:46
New changeset d1777515f9f53b452a4231d68196a7c0e5deb879 by Erlend Egeberg Aasland in branch 'main':
bpo-45138: Expand traced SQL statements in `sqlite3` trace callback (GH-28240)
https://github.com/python/cpython/commit/d1777515f9f53b452a4231d68196a7c0e5deb879
msg414785 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2022-03-09 02:47
Thanks for the contribution!
msg414787 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2022-03-09 08:40
Reopening bco. broken buildbots.
msg414793 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2022-03-09 11:14
Ah, one of my very first contributions, bpo-40318, comes back to haunt me:

In bpo-40318, we migrated from the old SQLite trace API (sqlite3_trace) to SQLite trace v2 API (sqlite3_trace_v2). GH-19581, which introduced this change, introduced a bug: the old trace API _implicitly_ expanded bound parameters; the new trace API does not. However, there was no tests for this behaviour, so the regression was unnoticed[^1].

So, this bpo is actually a bug fix; not a feature. It should be backported to 3.10, which contains the regression.

I'm preparing a fix for GH-28240, and I'll prepare a 3.10 backport including both GH-2840 and its upcoming fix.


[^1]: There has been no bug reports regarding this change in behaviour, so it seems to have gone under most people's radar.
msg414806 - (view) Author: miss-islington (miss-islington) Date: 2022-03-09 17:40
New changeset e801e88744f34508aa338f9f7f3f3baee012f813 by Erlend Egeberg Aasland in branch 'main':
bpo-45138: Revert GH-28240: Expand traced SQL statements (GH-31788)
https://github.com/python/cpython/commit/e801e88744f34508aa338f9f7f3f3baee012f813
History
Date User Action Args
2022-04-11 14:59:49adminsetgithub: 89301
2022-03-09 17:40:07miss-islingtonsetnosy: + miss-islington
messages: + msg414806
2022-03-09 17:12:37erlendaaslandsetpull_requests: + pull_request29892
2022-03-09 12:12:08erlendaaslandsetstage: resolved -> patch review
pull_requests: + pull_request29888
2022-03-09 11:14:46erlendaaslandsetmessages: + msg414793
2022-03-09 08:40:32erlendaaslandsetstatus: closed -> open

messages: + msg414787
2022-03-09 02:47:08JelleZijlstrasetstatus: open -> closed
resolution: fixed
messages: + msg414785

stage: patch review -> resolved
2022-03-09 02:46:44JelleZijlstrasetnosy: + JelleZijlstra
messages: + msg414784
2021-09-08 20:40:27erlendaaslandsettitle: [sqlite3] expand bound values in traced statements if possible -> [sqlite3] expand bound values in traced statements when possible
2021-09-08 14:19:47erlendaaslandsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26660
2021-09-08 14:05:14erlendaaslandcreate