This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author neologix
Recipients BreamoreBoy, flub, loewis, neologix, pitrou, sable, tim.peters
Date 2011-04-26.17:47:58
SpamBayes Score 2.220446e-16
Marked as misclassified No
Message-id <BANLkTinGgGdVr0J7KjrAF4hpKtnffgop_A@mail.gmail.com>
In-reply-to <1303829539.1.0.983983549419.issue3526@psf.upfronthosting.co.za>
Content
> it is possible to impact the memory allocation system on AIX using some environment variables (MALLOCOPTIONS and others)

LD_PRELOAD won't impact AIX's malloc behaviour, but allows you to
replace it transparently by any other implementation you like
(dlmalloc, ptmalloc, ...), without touching neither cpython nor your
application.

For example, let's says I want a Python version where getpid always returns 42.

$ cat /tmp/pid.c
int getpid(void)
{
        return 42;
}

$ gcc -o /tmp/pid.so /tmp/pid.c -fpic -shared

Now,

$ LD_PRELOAD=/tmp/pid.so python -c 'import os; print(os.getpid())'
42

That's it. If you replace pid.so by dlmalloc.so, you'll be using
dlmalloc instead of AIX's malloc, without having modified a single
line of code.
If you're concerned with impacting other applications, then you could
do something like:

$ cat python.c
#include <stdlib.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
        setenv("LD_PRELOAD", "/tmp/pid.so", 1);
        execvl(<path to real python>, argv);

        return 1;
}

And then:
$ ./python -c 'import os; print(os.getpid())'
42

> Also note that dlmalloc (or a derivative - ptmalloc) is part of GNU glibc which is used by most Linux systems, and is what you get when you call malloc.
> http://en.wikipedia.org/wiki/Malloc#dlmalloc_and_its_derivatives
>

Actually, glibc/eglibc versions have diverged quite a lot from the
original ptmalloc2, see for example http://bugs.python.org/issue11849
(that's one reason why embedding such a huge piece of code into Python
is probably not a good idea as highlighted by Antoine, it's updated
fairly frequently).

> So by using dlmalloc on SunOS and AIX you would get the same level of performance for memory operations that you already probably can appreciate on Linux systems.

Yes, but with the above "trick", you can do that without patching
python nor your app.
I mean, if you start embedding malloc in python, why stop there, and
not embed the whole glibc ;-)
Note that I realize this won't solve the problem for other AIX users
(if there are any left :-), but since this patch doesn't seem to be
gaining adhesion, I'm just proposing an alternative that I find
cleaner, simpler and easier to maintain.
History
Date User Action Args
2011-04-26 17:47:59neologixsetrecipients: + neologix, tim.peters, loewis, pitrou, sable, flub, BreamoreBoy
2011-04-26 17:47:58neologixlinkissue3526 messages
2011-04-26 17:47:58neologixcreate