This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author sable
Recipients loewis, pitrou, sable, tim.peters
Date 2008-09-09.15:58:42
SpamBayes Score 2.172902e-07
Marked as misclassified No
Message-id <1220975947.53.0.156358949337.issue3526@psf.upfronthosting.co.za>
In-reply-to
Content
Here is a new patch so that pymalloc can be combined with dlmalloc.

I first added the --with-pymalloc-mmap option to configure.in which
ensures that pymalloc arenas are allocated through mmap when possible.

However I found this was not enough: PyObject_Malloc uses arenas only
when handling objects smaller than 256 bytes. For bigger objects, it
directly rely on the system malloc. There are also some big buffers
which can be directly allocated through PyMem_MALLOC.

This patch can be activated by compiling Python with:
--with-pymalloc --with-pymalloc-mmap --with-dlmalloc

The behavior is then like that:
* PyObject_MALLOC will allocate arenas with mmap

* when allocating an object smaller than 256 bytes with 
PyObject_MALLOC, it will be stored in an arena (like before)

* when allocating an object bigger than 256 bytes with PyObject_MALLOC,
it will be allocated by dlmalloc (if it is smaller than 256KB it will go
in a dlmalloc pool, otherwise it will be mmaped)

* allocation through PyMem_MALLOC is handled by dlmalloc

I think it is a good compromise:
On systems like Linux, where the system malloc is already clever enough,
compiling with only --with-pymalloc should behave like before. On
systems like SunOS and AIX, this patch ensures that Python can benefit
of the speed of pymalloc for small objects, while ensuring that most of
the memory allocated can be correctly released at the system level.
History
Date User Action Args
2008-09-09 15:59:07sablesetrecipients: + sable, tim.peters, loewis, pitrou
2008-09-09 15:59:07sablesetmessageid: <1220975947.53.0.156358949337.issue3526@psf.upfronthosting.co.za>
2008-09-09 15:59:06sablelinkissue3526 messages
2008-09-09 15:59:06sablecreate