This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author kristjan.jonsson
Recipients kristjan.jonsson, larry, neologix, pitrou, tim.peters, vstinner
Date 2014-04-16.18:01:58
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1397671318.76.0.953355488687.issue21220@psf.upfronthosting.co.za>
In-reply-to
Content
Antoine:  The location of the arenas when they're individually allocated with mmap does not matter, no, but preferring to keep low address ones reduces vmem fragmentation, since they end up being clustered together in memory.  For the usable-arenas list, there is no extra O(n) because they were ordered anyway.  the effect of ARENA_STRATEGY is minor, but it helps for it to be consistent with POOL_STRATEGY.

The real win however is with POOL_STRATEGY.  Fragmentation is dramatically reduced.  This is demonstrated with the tools/scripts/memcrunch.py which you can use to experiment with it.  Performance e.g. of unittests also goes up.  The fact that there is a new O(n) sort operation when a pool becomes 'used' does not seem to matter for that.

Victor: I've tested using windows LFH many times before, the python obmalloc generally is much faster than that.  Annoying :).  It is actually a very good allocator.

The innovation here is the "lowest address strategy" which I have never seen before (it might be known, but then I'm not a CS) but is one that I have experimented with for often in the past.  It is suprisingly effective.  When there is memory churn, memory usage tends to migrate towards low addresses and free up memory.  Go ahead, try the scripts and see what happens.  The proof is in the pudding :)
History
Date User Action Args
2014-04-16 18:01:58kristjan.jonssonsetrecipients: + kristjan.jonsson, tim.peters, pitrou, vstinner, larry, neologix
2014-04-16 18:01:58kristjan.jonssonsetmessageid: <1397671318.76.0.953355488687.issue21220@psf.upfronthosting.co.za>
2014-04-16 18:01:58kristjan.jonssonlinkissue21220 messages
2014-04-16 18:01:58kristjan.jonssoncreate