Author vstinner
Recipients amaury.forgeotdarc, arigo, ghaering, iritkatriel, jcea, lkraav, pablogsal, pitrou, pxd, vstinner
Date 2021-02-15.17:21:37
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1613409697.9.0.0402175090192.issue15108@roundup.psfhosted.org>
In-reply-to
Content
The general issue here is a the PyTuple_New() is unsafe: it immediately tracks the newly created tuple in the GC, whereas the tuple is not initialized yet. If the GIL is released before the tuple is fully populated and something access to this tuple via the GC (ex: gc.get_objects()), accessing the tuple can crash, especially in the Python land (for example, repr(the_tuple) is likely to crash).

IMO the unsafe PyTuple_New() API should be avoided. For example, allocate an array of PyObject* on the stack memory, and then call _PyTuple_FromArray(). This API is safe because it only tracks the tuple once it's fully initialized, and it calls INCREF on items. Problem: this safe and efficient API is currently private.

There are other safe alternatives like Py_BuildValue("(OOO)", item1, item2, item3).

_pysqlite_fetch_one_row() calls PyTuple_New() and releases the GIL at each sqlite3_column_type() call, so yeah, it has this exact bug. By the way, it doesn't check for PyTuple_SetItem() failure, whereas it's currently possible that there is more than one strong reference to the tuple which is being populated (because of the GC issue).

PyTuple_New() is ok-ish if there is no way to trigger a GC collection and if the GIL cannot be released until the tuple is fully initialized.

Maybe we need a private _PyTuple_NewUntracked() API to create a tuple which is not tracked by the GC, and also a _PyTuple_ResizeUntracked() API. By the way, _PyTuple_Resize() sounds like a nonsense since a tuple is supposed to be immutable ;-)
History
Date User Action Args
2021-02-15 17:21:37vstinnersetrecipients: + vstinner, arigo, jcea, ghaering, amaury.forgeotdarc, pitrou, lkraav, pxd, pablogsal, iritkatriel
2021-02-15 17:21:37vstinnersetmessageid: <1613409697.9.0.0402175090192.issue15108@roundup.psfhosted.org>
2021-02-15 17:21:37vstinnerlinkissue15108 messages
2021-02-15 17:21:37vstinnercreate