This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: obmalloc radix tree typo in code
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: methane Nosy List: methane, miss-islington, nascheme
Priority: high Keywords: patch

Created on 2021-10-19 05:37 by nascheme, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29051 merged nascheme, 2021-10-19 05:57
PR 29122 merged miss-islington, 2021-10-21 14:11
Messages (5)
msg404268 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2021-10-19 05:37
There is a typo in the radix tree obmalloc code, spotted by Inada Naoki. 

-#define MAP_TOP_MASK (MAP_BOT_LENGTH - 1)
+#define MAP_TOP_MASK (MAP_TOP_LENGTH - 1)

This should be fixed both in the main branch and in 3.10.x.
msg404269 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-10-19 05:45
When I am trying to understand this issue, I see this segfault.

https://gist.github.com/methane/1b83e2abc6739017e0490c5f70a27b52

I am not sure this segfault is caused by this issue or not. If this is unrelated, I will create another issue.
msg404351 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2021-10-19 20:52
I have not yet been able to reproduce methane's crash.  My guess it it's not related.

An explanation of what I think the impact of this bug is:

The radix tree is used to determine if memory is from obmalloc or from the system malloc (i.e return value from address_in_range()).  WIth ADDRESS_BITS set to 48, we ignore the top 16 bits of addresses.  The next 10 bits are supposed to be the index into the top level node array for the radix tree.  Due to the bug, we mask those and only use the bottom 8 of those 10.  So, if you have virtual addresses that span more than that 8 bit range, we will index into the wrong node.  That means address_in_range() could give the wrong answer.  Which means you might try to free memory with the wrong malloc.

I think this is likely to be triggered only if you allocate a massive amount of memory, like 70 TB.  However, triggering it would depend on how the kernel maps virtual memory to the Python process.  I.e. there might be a wierd OS that gives pages at 0x7f0000000000 and then right after pages at 0x3f0000000000.
msg404602 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2021-10-21 13:38
New changeset 311910b31a4bd94dc79298388b7cb65ca5546438 by Neil Schemenauer in branch 'main':
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051)
https://github.com/python/cpython/commit/311910b31a4bd94dc79298388b7cb65ca5546438
msg404610 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2021-10-21 15:40
New changeset 1cdac61065e72db60d26e03ef9286d2743d7000e by Miss Islington (bot) in branch '3.10':
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051) (GH-29122)
https://github.com/python/cpython/commit/1cdac61065e72db60d26e03ef9286d2743d7000e
History
Date User Action Args
2022-04-11 14:59:51adminsetgithub: 89684
2021-10-21 15:41:32naschemesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-10-21 15:40:44naschemesetmessages: + msg404610
2021-10-21 14:11:29miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27400
2021-10-21 13:38:40naschemesetmessages: + msg404602
2021-10-19 20:52:59naschemesetmessages: + msg404351
2021-10-19 05:57:45naschemesetkeywords: + patch
pull_requests: + pull_request27322
2021-10-19 05:45:28methanesetmessages: + msg404269
2021-10-19 05:37:58naschemecreate