Message263968
> Julian Taylor added the comment:
>
> it defaulted to 128kb ten years ago, its a dynamic threshold since ages.
Indeed, and that's what encouraged switching the allocator to use mmap.
The problem with dynamic mmap threshold is that since the Python
allocator uses fixed-size arenas, basically malloc always ends up
allocating from the heap (brk).
Which means that given that we don't use a - compacting - garbage
collector, after a while the heap would end up quite fragmented, or
never shrink: for example let's say you allocate 1GB - on the heap -
and then you free them, but a single object is allocated at the top
of the heap, you heap never shrinks back.
This has bitten people (and myself a couple times at work).
Now, I see several options:
- revert to using malloc, but this will re-introduce the original problem
- build some form of hysteresis in the arena allocation
- somewhat orthogonally, I'd be interested to see if we couldn't
increase the arena size |
|
Date |
User |
Action |
Args |
2016-04-22 06:56:08 | neologix | set | recipients:
+ neologix, pitrou, vstinner, StyXman, jtaylor, dw, ztane, bar.harel |
2016-04-22 06:56:08 | neologix | link | issue26601 messages |
2016-04-22 06:56:08 | neologix | create | |
|