classification
Title: [sqlite3] use unraisable exceptions in callbacks
Type: Stage: patch review
Components: Extension Modules Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: erlendaasland, pablogsal, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2021-11-17 09:42 by erlendaasland, last changed 2021-11-29 15:22 by pablogsal.

Pull Requests
URL Status Linked Edit
PR 29591 merged erlendaasland, 2021-11-17 09:47
Messages (2)
msg406459 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-11-17 09:42
In order to print tracebacks from exceptions in SQLite callbacks, the sqlite3 extension module provides sqlite3.enable_callback_tracebacks(flag). Setting the flag to True instructs the sqlite3 extension module to PyErr_Print() if an exception occurs during a callback. Else, PyErr_Clear() is called.

From the sqlite3.enable_callback_tracebacks() docs:

    By default you will not get any tracebacks in user-defined functions,
    aggregates, converters, authorizer callbacks etc. If you want to debug
    them, you can call this function with flag set to True. Afterwards, you
    will get tracebacks from callbacks on sys.stderr. Use False to disable the
    feature again.


Few other exceptions use a similar approach:

    $ grep -r PyErr_Print Modules 
    Modules/_tkinter.c:        PyErr_Print();
    Modules/_testcapimodule.c:        PyErr_Print();
    Modules/main.c:    PyErr_Print();
    Modules/main.c:        PyErr_Print();
    Modules/main.c:        PyErr_Print();
    Modules/_io/bytesio.c:        PyErr_Print();
    Modules/_sqlite/connection.c:        PyErr_Print();
    Modules/_sqlite/cursor.c:                    PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_xxtestfuzz/fuzzer.c:        PyErr_Print();
    Modules/_ctypes/callbacks.c:    PyErr_Print();


We get a higher hit for unraisable exceptions:

    $ grep -r PyErr_WriteUnraisable Modules | wc -l
      45


AFAICS, using unraisable exceptions is a better approach.


Current behaviour:

    Python 3.10.0 (v3.10.0:b494f5935c, Oct  4 2021, 14:59:20) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sqlite3
    >>> cx = sqlite3.connect(":memory:")
    >>> cx.set_trace_callback(lambda stmt: 5/0)
    >>> cx.execute("select 1")
    <sqlite3.Cursor object at 0x109bba2c0>
    >>> sqlite3.enable_callback_tracebacks(True)
    >>> cx.execute("select 1")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <lambda>
    ZeroDivisionError: division by zero
    <sqlite3.Cursor object at 0x1099046c0>


With unraisable exceptions:

    Python 3.11.0a2+ (heads/sqlite-unraisable-exceptions-dirty:de29590d6a, Nov 17 2021, 10:29:19) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sqlite3
    >>> cx = sqlite3.connect(":memory:")
    >>> cx.set_trace_callback(lambda stmt: 5/0)
    >>> cx.execute("select 1")
    <sqlite3.Cursor object at 0x10b1fe720>
    >>> sqlite3.enable_callback_tracebacks(True)
    >>> cx.execute("select 1")
    Exception ignored in: <function <lambda> at 0x10b4e3ee0>
    Traceback (most recent call last):
      File "<stdin>", line 1, in <lambda>
    ZeroDivisionError: division by zero
    <sqlite3.Cursor object at 0x10b1fe840>


The user experience is mostly unchanged; we get one extra line, telling us that the exception was ignored. Also, users can now use sys.unraisablehook:

    >>> sys.unraisablehook = lambda unraisable: print(unraisable)
    >>> cx.execute("select 1")
    UnraisableHookArgs(exc_type=<class 'ZeroDivisionError'>,     exc_value=ZeroDivisionError('division by zero'), exc_traceback=<traceback object at 0x10b559900>, err_msg=None, object=<function <lambda> at 0x10b4e3ee0>)
    <sqlite3.Cursor object at 0x10b1fe840>


The only question I have, is if we should deprecate sqlite3.enable_callback_tracebacks() after switching to unraisable exceptions.
msg407285 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-11-29 15:22
New changeset c4a69a4ad035513ada1c0d41a46723606b538e13 by Erlend Egeberg Aasland in branch 'main':
bpo-45828: Use unraisable exceptions within sqlite3 callbacks (FH-29591)
https://github.com/python/cpython/commit/c4a69a4ad035513ada1c0d41a46723606b538e13
History
Date User Action Args
2021-11-29 15:22:36pablogsalsetnosy: + pablogsal
messages: + msg407285
2021-11-17 09:47:14erlendaaslandsetkeywords: + patch
stage: patch review
pull_requests: + pull_request27834
2021-11-17 09:44:48erlendaaslandsetnosy: + serhiy.storchaka
2021-11-17 09:42:19erlendaaslandcreate