Message383251
Thanks, I will see if I can get additional diagnostics today. Otherwise I will have to start recompiling Python with diagnostic hooks. I have never had to rebuild python on windows so it may take a while to figure out how to make progress.
The monkey patch is less scary than it looks. Since I only need these slots of specific basic types that I designate (not user selected), the extra types all extend without altering the underlaying type, and the class layout is controlled by a meta class so that only Java can create these type objects, it is reasonably robust. However, yes changes in Python can break it. It is basically an attempt to create a mixin for concrete classes without the benefit of a dictoffset type slot. It took about a month do develop something that safely added the slot without overrunning, leaking or creating other memory issues (and played nice when Python added its own dict slot).
Unfortunately, as I do many 100s on JSlot tests during method resolution using a dict based approach was unacceptably slow. I would love if there was a formal method to create multiple inheritance mixins for Python basic types, but that is another issue.
```
/**
* Allocate a new Python object with a slot for Java.
*
* We need extra space to store our values, but because there
* is no way to do so without disturbing the object layout.
* Fortunately, Python already handles this for dict and weakref.
* Python aligns the ends of the structure and increases the
* base type size to add additional slots to a standard object.
*
* We will use the same trick to add an additional slot for Java
* after the end of the object outside of where Python is looking.
* As the memory is aligned this is safe to do. We will use
* the alloc and finalize slot to recognize which objects have this
* extra slot appended.
*/
PyObject* PyJPValue_alloc(PyTypeObject* type, Py_ssize_t nitems )
{
// Modification from Python to add size elements
const size_t size = _PyObject_VAR_SIZE(type, nitems + 1) + sizeof (JPValue);
PyObject *obj = (PyType_IS_GC(type)) ? _PyObject_GC_Malloc(size)
: (PyObject *) PyObject_MALLOC(size);
if (obj == NULL)
return PyErr_NoMemory(); // GCOVR_EXCL_LINE
memset(obj, 0, size);
Py_ssize_t refcnt = ((PyObject*) type)->ob_refcnt;
if (type->tp_itemsize == 0)
PyObject_Init(obj, type);
else
PyObject_InitVar((PyVarObject *) obj, type, nitems);
// This line is required to deal with Python bug (GH-11661)
// Some versions of Python fail to increment the reference counter of
// heap types properly.
if (refcnt == ((PyObject*) type)->ob_refcnt)
Py_INCREF(type); // GCOVR_EXCL_LINE
if (PyType_IS_GC(type))
{
PyObject_GC_Track(obj);
}
return obj;
}
bool PyJPValue_hasJavaSlot(PyTypeObject* type)
{
if (type == NULL
|| type->tp_alloc != (allocfunc) PyJPValue_alloc
|| type->tp_finalize != (destructor) PyJPValue_finalize)
return false; // GCOVR_EXCL_LINE
return true;
}
// Check for a Java slot (must work on all object types)
Py_ssize_t PyJPValue_getJavaSlotOffset(PyObject* self)
{
PyTypeObject *type = Py_TYPE(self);
if (type == NULL
|| type->tp_alloc != (allocfunc) PyJPValue_alloc
|| type->tp_finalize != (destructor) PyJPValue_finalize)
return 0;
Py_ssize_t offset;
Py_ssize_t sz = Py_SIZE(self);
// I have no clue what negative sizes mean
if (sz < 0)
sz = -sz;
if (type->tp_itemsize == 0)
offset = _PyObject_VAR_SIZE(type, 1);
else
offset = _PyObject_VAR_SIZE(type, sz + 1);
return offset;
}
JPValue* PyJPValue_getJavaSlot(PyObject* self)
{
Py_ssize_t offset = PyJPValue_getJavaSlotOffset(self);
if (offset == 0)
return NULL;
JPValue* value = (JPValue*) (((char*) self) + offset);
if (value->getClass() == NULL)
return NULL;
return value;
}
void PyJPValue_free(void* obj)
{
// Normally finalize is not run on simple classes.
PyTypeObject *type = Py_TYPE(obj);
if (type->tp_finalize != NULL)
type->tp_finalize((PyObject*) obj);
if (type->tp_flags & Py_TPFLAGS_HAVE_GC)
PyObject_GC_Del(obj);
else
PyObject_Free(obj); // GCOVR_EXCL_LINE
}
void PyJPValue_finalize(void* obj)
{
JPValue* value = PyJPValue_getJavaSlot((PyObject*) obj);
... destroy the objects hidden slot if it exists
}
``` |
|
Date |
User |
Action |
Args |
2020-12-17 16:23:01 | Thrameos | set | recipients:
+ Thrameos, paul.moore, tim.golden, zach.ware, steve.dower |
2020-12-17 16:23:01 | Thrameos | set | messageid: <1608222181.06.0.954022943882.issue42529@roundup.psfhosted.org> |
2020-12-17 16:23:01 | Thrameos | link | issue42529 messages |
2020-12-17 16:22:59 | Thrameos | create | |
|