Author vstinner
Recipients pitrou, rhettinger, tim.peters, vstinner
Date 2013-08-26.10:52:45
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1377514366.21.0.00190556268388.issue18835@psf.upfronthosting.co.za>
In-reply-to
Content
The default allocator for PyObject is PyType_GenericAlloc(). If the type has the Py_TPFLAGS_HAVE_GC flag, PyType_GenericAlloc() calls _PyObject_GC_Malloc(). It is the case for the set type. _PyObject_GC_Malloc() adds an header of sizeof(PyGC_Head) (12 bytes on x86) before the PyObject data and then calls PyObject_MALLOC(). By default, Python uses pymalloc for PyObject_MALLOC() which uses an alignment of 8 bytes.

For set, I suppose that you are talking about the allocation of the "table", not of the set object itself. set_table_resize() uses PyMem_NEW() to allocate the table.

If we provide a PyMem_MallocAligned(alignment, size), table entries would be aligned, but not entries of the smalltable. Does it matter?

> I suggest variants such as PyMem_Alloc32(n) and PyMem_Alloc64(n) to return suitably aligned data blocks.

I would prefer a parameter rather than a new function per alignment! Which API does you propose exactly? On Linux, I see at least 3 functions:

int posix_memalign(void **memptr, size_t alignment, size_t size);
void *valloc(size_t size);
void *memalign(size_t boundary, size_t size);

Do you propose aligned variant for PyMem, PyMem_Raw and PyObject, or only PyMem?

"Unless the memory allocator actually supports it, this means you lose a whole lot of memory for padding, though... Memory which will sit there unused at the end of another cacheline."

What is the alignment of a cacheline? Can a line starts at any address?

Do you have an idea of performance benefit of memory alignment?

Adding yet another API to allocate memory has a cost.

Python 3.4 already implemented the PEP 445 which added many new functions. If we add new functions, they should conform the PEP 445 (need get/set allocator functions to support hooks to track the memory usage).
History
Date User Action Args
2013-08-26 10:52:46vstinnersetrecipients: + vstinner, tim.peters, rhettinger, pitrou
2013-08-26 10:52:46vstinnersetmessageid: <1377514366.21.0.00190556268388.issue18835@psf.upfronthosting.co.za>
2013-08-26 10:52:46vstinnerlinkissue18835 messages
2013-08-26 10:52:45vstinnercreate