Author vstinner
Recipients vstinner
Date 2021-04-08.08:29:01
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1617870542.89.0.74089968947.issue43770@roundup.psfhosted.org>
In-reply-to
Content
I tried to put assertions in PyObject_GetAttr(), _PyObject_LookupAttr(), _PyObject_GetMethod() and PyObject_SetAttr() to ensure that the type is ready. It breaks many tests, examples:

    test_buffer test_pickle test_picklebuffer test_pickletools
    test_structmembers

For example, Modules/_testbuffer.c does not initialize its ndarray type.

Example of crash:
-----------------
Objects/object.c:892: PyObject_GetAttr: Assertion "tp->tp_flags & (1UL << 12)" failed
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x7f4d89119a00
object refcount : 4
object type     : 0x8768c0
object type name: type
object repr     : <class 'ndarray'>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007f4d96eba740 (most recent call first):
  File "/home/vstinner/python/master/Lib/test/test_picklebuffer.py", line 80 in test_ndarray_2d
  (...)

Extension modules: _testcapi, _testbuffer (total: 2)
-----------------

Another example of crash:
-----------------
Objects/object.c:892: PyObject_GetAttr: Assertion "tp->tp_flags & (1UL << 12)" failed
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x7f174ec9aa00
object refcount : 3
object type     : 0x8768c0
object type name: type
object repr     : <class 'ndarray'>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007f175ca42740 (most recent call first):
  File "/home/vstinner/python/master/Lib/test/pickletester.py", line 304 in __reduce_ex__
  (...)

Extension modules: _testcapi, _testbuffer, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg.lapack_lite, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator (total: 16)
-----------------


The C API tries to make the type ready automatically. For example, PyObject_Hash() calls PyType_Ready() if needed and then retry tp_hash:

Py_hash_t
PyObject_Hash(PyObject *v)
{
    PyTypeObject *tp = Py_TYPE(v);
    if (tp->tp_hash != NULL)
        return (*tp->tp_hash)(v);
    /* To keep to the general practice that inheriting
     * solely from object in C code should work without
     * an explicit call to PyType_Ready, we implicitly call
     * PyType_Ready here and then check the tp_hash slot again
     */
    if (tp->tp_dict == NULL) {
        if (PyType_Ready(tp) < 0)
            return -1;
        if (tp->tp_hash != NULL)
            return (*tp->tp_hash)(v);
    }
    /* Otherwise, the object can't be hashed */
    return PyObject_HashNotImplemented(v);
}
History
Date User Action Args
2021-04-08 08:29:02vstinnersetrecipients: + vstinner
2021-04-08 08:29:02vstinnersetmessageid: <1617870542.89.0.74089968947.issue43770@roundup.psfhosted.org>
2021-04-08 08:29:02vstinnerlinkissue43770 messages
2021-04-08 08:29:01vstinnercreate