classification
Title: Fix all invalid usage of borrowed references
Type: crash Stage:
Components: Interpreter Core, Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: arigo, vstinner
Priority: normal Keywords:

Created on 2012-03-20 12:43 by vstinner, last changed 2013-01-03 02:48 by vstinner. This issue is now closed.

Messages (3)
msg156401 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-03-20 12:43
I fixed the issues #14211 and #14231. These issues were invalid usage of borrowed reference: the reference can be destroyed before the reference is used.

Armin Rigo replied:

"I will attempt a last time to mention that the docstrings in borrowed_ref_*.py used to say they were *examples*.

That means: (1) find any internal or external C function that returns a borrowed reference; (2) find all callers and write down all the places that don't immediately either Py_INCREF() the returned value or immediately forget about it; (3) for each such place, either come up painfully with a complicated explanation of why it's safe in all possible cases, or in doubt, just fix it by adding Py_INCREF()/Py_DECREF().

What I did in writing these two borrowed_ref_*.py was to do instead (3') spend a few hours figuring out how to exploit the issue until we get a segfault.  I did it for two examples, but what I'm definitely not going to do is spend N times a few hours for a large number N.  If python-dev people just fix the two examples, remove the crashers, and just forget about the issue, then well, the point is missed, but I'm not going to fight it."

Sorry Armin, I didn't know that the issue of borrowed was a generic issue and more code need to be fixed. So I'm opening this issue to track of invalid usage of borrowed references.
msg156403 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-03-20 12:53
> (1) find any internal or external C function that returns a borrowed reference

Doc/data/refcounts.dat can be used for that:

PyCell_GET
PyDict_GetItem
PyDict_GetItemString
PyErr_Occurred
PyEval_GetBuiltins
PyEval_GetLocals
PyEval_GetGlobals
PyEval_GetFrame
PyFile_Name
PyFunction_GetClosure
PyFunction_GetCode
PyFunction_GetDefaults
PyFunction_GetGlobals
PyFunction_GetModule
Py_InitModule
Py_InitModule3
Py_InitModule4
PyImport_AddModule
PyImport_GetModuleDict
PyList_GET_ITEM
PyList_GetItem
PyMethod_Class
PyMethod_Function
PyMethod_GET_CLASS
PyMethod_GET_FUNCTION
PyMethod_GET_SELF
PyMethod_Self
PyModule_GetDict
PyObject_Init
PySequence_Fast_GET_ITEM
PySys_GetObject
PySys_GetXOptions
PyThreadState_GetDict
PyTuple_GET_ITEM
PyTuple_GetItem
PyWeakref_GET_OBJECT
PyWeakref_GetObject

(this list may be incomplete, it should be checked)

Not affected (always return NULL):

PyErr_NoMemory
PyErr_Set...
msg178906 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-01-03 02:48
Nobody looks motivated to work on such complex and long issue, so I close the issue. It's maybe simpler to open an issue per invalid usage of borrowed reference (as it was done for #14211 and #14231).
History
Date User Action Args
2013-01-03 02:48:28vstinnersetstatus: open -> closed
resolution: wont fix
messages: + msg178906
2012-03-20 12:54:00vstinnersetmessages: + msg156403
2012-03-20 12:45:36vstinnersetnosy: + arigo
2012-03-20 12:43:43vstinnercreate