This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author neologix
Recipients BreamoreBoy, flub, loewis, neologix, pitrou, sable, tim.peters
Date 2011-07-26.18:31:50
SpamBayes Score 1.7625245e-08
Marked as misclassified No
Message-id <1311705111.02.0.489252114401.issue3526@psf.upfronthosting.co.za>
In-reply-to
Content
> Fortunately, it is easy to solve by defining the following in
> dlmalloc:
> #define HAVE_MORECORE 0

I was expecting this answer ;-)
Here's a quick demo, on a Linux box:

cf@neobox:~/cpython$ ./python Tools/pybench/pybench.py -n 1
-------------------------------------------------------------------------------
Totals:                          19787ms  19787ms

cf@neobox:~/cpython$ MALLOC_MMAP_THRESHOLD_=0 ./python Tools/pybench/pybench.py -n 1
[...]
-------------------------------------------------------------------------------
Totals:                          33375ms  33375ms

That's a mere 70% slowdown, and without pymalloc, it would be much worse. malloc with mmap() is way slower than with sbrk() (see http://sources.redhat.com/ml/libc-alpha/2006-03/msg00033.html for more details). Since your benchmarks don't show this type of regression it probably means that AIX's malloc implementation is really broken (there's also the fact that part of the allocations are still routed to the libc's malloc, or maybe your workload is too specific to demonstrate this behavior).

> sbrk is generally considered quite archaic.

I wouldn't say that; see the above link on malloc's dynamic mmap() threshold.

> I also don't expect this patch to be integrated spontaneously without
> someone interested in AIX pushing for it.

Indeed.
As far as I'm concerned, there are two "showstoppers":
- shipping an implementation of dlmalloc with Python
- mixing dlmalloc with the host's malloc implementation

But I think the main problem with this patch is that AIX represents such a tiny fraction of the user base. This might change in the future, especially if IBM is successfull in its effort of pushing AIX (I hope they'll finally fix AIX's malloc by then...).

> I have been quite busy recently with the birth of my second daughter,
> a new job, a new home town and soon a new home.

Congratulations, and good luck!
History
Date User Action Args
2011-07-26 18:31:51neologixsetrecipients: + neologix, tim.peters, loewis, pitrou, sable, flub, BreamoreBoy
2011-07-26 18:31:51neologixsetmessageid: <1311705111.02.0.489252114401.issue3526@psf.upfronthosting.co.za>
2011-07-26 18:31:50neologixlinkissue3526 messages
2011-07-26 18:31:50neologixcreate