This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author sable
Recipients loewis, pitrou, sable
Date 2008-09-03.10:28:07
SpamBayes Score 3.1391253e-05
Marked as misclassified No
Message-id <1220437690.76.0.371830111912.issue3526@psf.upfronthosting.co.za>
In-reply-to
Content
[sorry for the late reply, I have been on holidays]

Martin:
you are right that this memory is moved to swap and does not consume any
"real" memory; however we decided to work on this patch because we
observed on our application some performances degradation due to this
memory not being deallocated correctly.

Since then we have done some quite extensive tests (with the help of a
consultant at Sun): they have shown that this unnecessary swapping has a
noticeable impact on performances and at worst, when the system memory
is saturated, can completely put a server on its knees for several
minutes (we're talking of top of the line SunOS and AIX servers with
hundreds of GB of memory).

I will write a complete document explaining the tests and observations
that we did, but this memory issue was critical for us given the
degradation of performances it was generating on our production servers.

Concerning dlmalloc, you are right that it would be cleaner to improve
obmalloc so that it uses mmap when necessary, instead of adding another
layer with dlmalloc (even though that is what actually currently happens
on linux systems where dlmalloc is integrated in libc).

I will try to do that patch in coming weeks (obmalloc mostly allocates
some 256KB arenas so it should nearly always use mmap).
History
Date User Action Args
2008-09-03 10:28:11sablesetrecipients: + sable, loewis, pitrou
2008-09-03 10:28:10sablesetmessageid: <1220437690.76.0.371830111912.issue3526@psf.upfronthosting.co.za>
2008-09-03 10:28:09sablelinkissue3526 messages
2008-09-03 10:28:07sablecreate