classification
Title: obmalloc: eliminate limit on pool size
Type: enhancement Stage: patch review
Components: Interpreter Core Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: tim.peters Nosy List: tim.peters
Priority: normal Keywords: patch

Created on 2019-06-10 00:36 by tim.peters, last changed 2019-06-10 04:25 by tim.peters.

Pull Requests
URL Status Linked Edit
PR 13934 open tim.peters, 2019-06-10 00:43
Messages (1)
msg345097 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2019-06-10 00:36
On 64-bit Python, many object sizes essentially doubled over 32-bit Python, because Python objects are so heavy with pointers.  More recently, forcing alignment to 16 bytes on 64-bit boxes boosted the memory requirements more modestly.

But obmalloc's 256 KiB arenas and 4 KiB pools haven't changed since obmalloc was first written, and its `address_in_range()` machinery cannot deal with pools bigger than that (they're segfault factories, because the machinery relies on that a pool is no larger than a system page).

obmalloc's fastest paths are those that stay within a pool.  Whenever a pool boundary is hit, it necessarily gets slower, then slower still if an arena boundary is hit.

So I propose to:

- Remove the 4 KiB pool limit, by making `address_in_range()` page-based rather than pool-based.  Pools should be able to span any power-of-2 number of pages.  Then a pool for a given size class will be able to hold that many more times as many objects too, and so stay in the fastest paths more often.

- On 64-bit boxes, increase both POOL_SIZE and ARENA_SIZE.
History
Date User Action Args
2019-06-10 04:25:22tim.peterssetassignee: tim.peters
2019-06-10 00:43:42tim.peterssetkeywords: + patch
stage: patch review
pull_requests: + pull_request13801
2019-06-10 00:36:19tim.peterscreate