This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Thrameos
Recipients Thrameos, paul.moore, steve.dower, tim.golden, zach.ware
Date 2020-12-17.16:22:59
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1608222181.06.0.954022943882.issue42529@roundup.psfhosted.org>
In-reply-to
Content
Thanks, I will see if I can get additional diagnostics today.  Otherwise I will have to start recompiling Python with diagnostic hooks.  I have never had to rebuild python on windows so it may take a while to figure out how to make progress.

The monkey patch is less scary than it looks.   Since I only need these slots of specific basic types that I designate (not user selected), the extra types all extend without altering the underlaying type, and the class layout is controlled by a meta class so that only Java can create these type objects, it is reasonably robust.   However, yes changes in Python can break it.  It is basically an attempt to create a mixin for concrete classes without the benefit of a dictoffset type slot.  It took about a month do develop something that safely added the slot without overrunning, leaking or creating other memory issues (and played nice when Python added its own dict slot). 

Unfortunately, as I do many 100s on JSlot tests during method resolution using a dict based approach was unacceptably slow.  I would love if there was a formal method to create multiple inheritance mixins for Python basic types, but that is another issue.


```
/**
 * Allocate a new Python object with a slot for Java.
 *
 * We need extra space to store our values, but because there
 * is no way to do so without disturbing the object layout.
 * Fortunately, Python already handles this for dict and weakref.
 * Python aligns the ends of the structure and increases the
 * base type size to add additional slots to a standard object.
 *
 * We will use the same trick to add an additional slot for Java
 * after the end of the object outside of where Python is looking.
 * As the memory is aligned this is safe to do.  We will use
 * the alloc and finalize slot to recognize which objects have this
 * extra slot appended.
 */
PyObject* PyJPValue_alloc(PyTypeObject* type, Py_ssize_t nitems )
{
	// Modification from Python to add size elements
	const size_t size = _PyObject_VAR_SIZE(type, nitems + 1) + sizeof (JPValue);
	PyObject *obj = (PyType_IS_GC(type)) ? _PyObject_GC_Malloc(size)
			: (PyObject *) PyObject_MALLOC(size);
	if (obj == NULL)
		return PyErr_NoMemory(); // GCOVR_EXCL_LINE
	memset(obj, 0, size);

	Py_ssize_t refcnt = ((PyObject*) type)->ob_refcnt;
	if (type->tp_itemsize == 0)
		PyObject_Init(obj, type);
	else
		PyObject_InitVar((PyVarObject *) obj, type, nitems);

	// This line is required to deal with Python bug (GH-11661)
	// Some versions of Python fail to increment the reference counter of
	// heap types properly.
	if (refcnt == ((PyObject*) type)->ob_refcnt)
		Py_INCREF(type);  // GCOVR_EXCL_LINE

	if (PyType_IS_GC(type))
	{
		PyObject_GC_Track(obj);
	}
	return obj;
}

bool PyJPValue_hasJavaSlot(PyTypeObject* type)
{
	if (type == NULL
			|| type->tp_alloc != (allocfunc) PyJPValue_alloc
			|| type->tp_finalize != (destructor) PyJPValue_finalize)
		return false;  // GCOVR_EXCL_LINE
	return true;
}

// Check for a Java slot (must work on all object types)
Py_ssize_t PyJPValue_getJavaSlotOffset(PyObject* self)
{
	PyTypeObject *type = Py_TYPE(self);
	if (type == NULL
			|| type->tp_alloc != (allocfunc) PyJPValue_alloc
			|| type->tp_finalize != (destructor) PyJPValue_finalize)
		return 0;
	Py_ssize_t offset;
	Py_ssize_t sz = Py_SIZE(self);
	// I have no clue what negative sizes mean
	if (sz < 0)
		sz = -sz;
	if (type->tp_itemsize == 0)
		offset = _PyObject_VAR_SIZE(type, 1);
	else
		offset = _PyObject_VAR_SIZE(type, sz + 1);
	return offset;
}

JPValue* PyJPValue_getJavaSlot(PyObject* self)
{
	Py_ssize_t offset = PyJPValue_getJavaSlotOffset(self);
	if (offset == 0)
		return NULL;
	JPValue* value = (JPValue*) (((char*) self) + offset);
	if (value->getClass() == NULL)
		return NULL;
	return value;
}

void PyJPValue_free(void* obj)
{
	// Normally finalize is not run on simple classes.
	PyTypeObject *type = Py_TYPE(obj);
	if (type->tp_finalize != NULL)
		type->tp_finalize((PyObject*) obj);
	if (type->tp_flags & Py_TPFLAGS_HAVE_GC)
		PyObject_GC_Del(obj);
	else
		PyObject_Free(obj);  // GCOVR_EXCL_LINE
}

void PyJPValue_finalize(void* obj)
{
	JPValue* value = PyJPValue_getJavaSlot((PyObject*) obj);
	... destroy the objects hidden slot if it exists
}

```
History
Date User Action Args
2020-12-17 16:23:01Thrameossetrecipients: + Thrameos, paul.moore, tim.golden, zach.ware, steve.dower
2020-12-17 16:23:01Thrameossetmessageid: <1608222181.06.0.954022943882.issue42529@roundup.psfhosted.org>
2020-12-17 16:23:01Thrameoslinkissue42529 messages
2020-12-17 16:22:59Thrameoscreate