classification
Title: Set ADDRESS_BITS to 64 for obmalloc radix tree
Type: enhancement Stage: resolved
Components: Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: methane Nosy List: lukasz.langa, methane, nascheme
Priority: normal Keywords: patch

Created on 2021-10-19 16:48 by nascheme, last changed 2021-10-21 21:06 by lukasz.langa. This issue is now closed.

Files
File name Uploaded Description Edit
pyperf-compare-addr-64.txt nascheme, 2021-10-19 16:48
Pull Requests
URL Status Linked Edit
PR 29062 merged nascheme, 2021-10-19 17:34
Messages (3)
msg404317 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2021-10-19 16:48
Given this feedback:

https://github.com/python/cpython/pull/14474/files#r725488766

it is perhaps not so safe to assume that only the lower 48 bits of virtual addresses are significant.  I had the idea that Go made similar assumptions but now I'm not sure it does.  There is also a comment in this article about 5-level page tables in Linux:

https://lwn.net/Articles/717293/
https://lwn.net/Articles/717300/

I.e. that Linux does not allocate virtual address space above 47-bit by default.

Setting ADDRESS_BITS to 64 is safer and the performance impact seems small. The virtual memory size of a small Python process goes up a little.  Resident set size doesn't significantly change.  I think the pyperformance changes are just noise.  The pyperformance attached file shows the 3.10 branch with ADDRESS_BITS set to 48 and to 64.
msg404661 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-10-21 21:05
New changeset 0224b7180b280794b9fba62057b278ffb536c86f by Neil Schemenauer in branch 'main':
bpo-45526: obmalloc radix use 64 addr bits (GH-29062)
https://github.com/python/cpython/commit/0224b7180b280794b9fba62057b278ffb536c86f
msg404662 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-10-21 21:06
Thanks, Neil! ✨ 🍰 ✨
History
Date User Action Args
2021-10-21 21:06:33lukasz.langasetstatus: open -> closed
type: enhancement
messages: + msg404662

resolution: fixed
stage: patch review -> resolved
2021-10-21 21:05:55lukasz.langasetnosy: + lukasz.langa
messages: + msg404661
2021-10-19 17:34:05naschemesetkeywords: + patch
pull_requests: + pull_request27329
2021-10-19 16:48:44naschemecreate