New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reference-counting problem in sqlite #81528
Comments
There are a couple of bugs in sqlite bindings have been found related to reference-counting. Long version: static PyObject *
Custom_set_callback(CustomObject *self, PyObject* args)
{
PyObject* display_str;
display_str = PyUnicode_FromFormat("set_callback called with cb=%R id=%i ob_refcnt=%i\n", args, args, args->ob_refcnt);
PyObject_Print(display_str, stdout, Py_PRINT_RAW);
if (PyDict_SetItem(self->function_pinboard, args, Py_None) == -1) return NULL;
//sqlite3_trace(self->db, _trace_callback, trace_callback);
self->callback_handler = args;
display_str = PyUnicode_FromFormat("set_callback done for cb=%R id=%i ob_refcnt=%i\n", args, args, args->ob_refcnt);
PyObject_Print(display_str, stdout, Py_PRINT_RAW);
Py_RETURN_NONE;
}
static PyObject *
Custom_call_callback(CustomObject *self)
{
PyObject* display_str;
display_str = PyUnicode_FromFormat("call with id=%i ob_refcnt=%i\n", self->callback_handler ,
self->callback_handler->ob_refcnt);
PyObject_Print(display_str, stdout, Py_PRINT_RAW);
Py_RETURN_NONE;
} Python code:
>>>> class TEST:
def log(self, msg=""):
pass
>>>> t = TEST()
>>>> conn = Custom()
>>>> conn.set_trace_callback(t.log)
set_callback called with cb=<bound method TEST.log of <__main__.TEST object at 0x10bc60128>> id=196094408 ob_refcnt=1
set_callback done for cb=<bound method TEST.log of <__main__.TEST object at 0x10bc60128>> id=196094408 ob_refcnt=2
>>>> conn.set_trace_callback(t.log)
set_callback called with cb=<bound method TEST.log of <__main__.TEST object at 0x10bc60128>> id=196095112 ob_refcnt=1
set_callback done for cb=<bound method TEST.log of <__main__.TEST object at 0x10bc60128>> id=196095112 ob_refcnt=1
conn.call()
call with id=196095112 ob_refcnt=0 After second conn.set_trace_callback(t.log) call, object t.log reference-count is not increased because 't.log in self->function_pinboard' returns True thus self->function_pinboard[t.log] is not replaced and t.log is not increfed, but it replaces old object in self->callback_handler. Also, there is no cleaning of self->function_pinboard. This leads to leaks every object passed as callback(see test_leak() in bug.py). |
Why not use sqlite3_create_function_v2 to register a destructor for the function that does Py_DECREF and dispense with the dictionaries entirely? |
Because destructor can be registered only for particular functions. For example sqlite3_progress_handler() can't register destructor. |
But there can be only one progress handler per connection, so you can just keep an single reference in the struct yourself. |
At my point of view, dererencing from sqlite destructor has next flaws:
Of cause, if it's necessary to use sqlite3_create_function_v2, I'm ready to make appropriate changes. |
The _sqlite3 module clearly is not free of bugs in this area. Binding the Python function to the exact lifetime of the sqlite function seems less likely to be buggy than replicating sqlite's function reference management independently. sqlite3_create_function_v2 was added in 2010; I think we can depend on it. Also, won't the callback pretty much be Py_DECREF(something)? |
I've pushed changes bce7fdc. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: