Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

obmalloc radix tree typo in code #89684

Closed
nascheme opened this issue Oct 19, 2021 · 5 comments
Closed

obmalloc radix tree typo in code #89684

nascheme opened this issue Oct 19, 2021 · 5 comments
Assignees
@methane
Labels
3.10 only security fixes 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@nascheme
Copy link
Member

BPO 45521
Nosy @nascheme, @methane, @miss-islington
PRs
  • bpo-45521: Fix a bug in the obmalloc radix tree code. #29051
  • [3.10] bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051) #29122
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/methane'
closed_at = <Date 2021-10-21.15:41:32.700>
created_at = <Date 2021-10-19.05:37:58.958>
labels = ['interpreter-core', '3.10', '3.11']
title = 'obmalloc radix tree typo in code'
updated_at = <Date 2021-10-21.15:41:32.700>
user = 'https://github.com/nascheme'

    bugs.python.org fields:

    activity = <Date 2021-10-21.15:41:32.700>
actor = 'nascheme'
assignee = 'methane'
closed = True
closed_date = <Date 2021-10-21.15:41:32.700>
closer = 'nascheme'
components = ['Interpreter Core']
creation = <Date 2021-10-19.05:37:58.958>
creator = 'nascheme'
dependencies = []
files = []
hgrepos = []
issue_num = 45521
keywords = ['patch']
message_count = 5.0
messages = ['404268', '404269', '404351', '404602', '404610']
nosy_count = 3.0
nosy_names = ['nascheme', 'methane', 'miss-islington']
pr_nums = ['29051', '29122']
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue45521'
versions = ['Python 3.10', 'Python 3.11']
    @nascheme
    Copy link
    Member Author

    There is a typo in the radix tree obmalloc code, spotted by Inada Naoki.

    -#define MAP_TOP_MASK (MAP_BOT_LENGTH - 1)
    +#define MAP_TOP_MASK (MAP_TOP_LENGTH - 1)

    This should be fixed both in the main branch and in 3.10.x.

    @nascheme nascheme added 3.10 only security fixes 3.11 only security fixes labels Oct 19, 2021
    @nascheme nascheme added interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.10 only security fixes 3.11 only security fixes labels Oct 19, 2021
    @nascheme nascheme added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Oct 19, 2021
    @methane
    Copy link
    Member

    methane commented Oct 19, 2021

    When I am trying to understand this issue, I see this segfault.

    https://gist.github.com/methane/1b83e2abc6739017e0490c5f70a27b52

    I am not sure this segfault is caused by this issue or not. If this is unrelated, I will create another issue.

    @nascheme
    Copy link
    Member Author

    I have not yet been able to reproduce methane's crash. My guess it it's not related.

    An explanation of what I think the impact of this bug is:

    The radix tree is used to determine if memory is from obmalloc or from the system malloc (i.e return value from address_in_range()). WIth ADDRESS_BITS set to 48, we ignore the top 16 bits of addresses. The next 10 bits are supposed to be the index into the top level node array for the radix tree. Due to the bug, we mask those and only use the bottom 8 of those 10. So, if you have virtual addresses that span more than that 8 bit range, we will index into the wrong node. That means address_in_range() could give the wrong answer. Which means you might try to free memory with the wrong malloc.

    I think this is likely to be triggered only if you allocate a massive amount of memory, like 70 TB. However, triggering it would depend on how the kernel maps virtual memory to the Python process. I.e. there might be a wierd OS that gives pages at 0x7f0000000000 and then right after pages at 0x3f0000000000.

    @nascheme
    Copy link
    Member Author

    New changeset 311910b by Neil Schemenauer in branch 'main':
    bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051)
    311910b

    @nascheme
    Copy link
    Member Author

    New changeset 1cdac61 by Miss Islington (bot) in branch '3.10':
    bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051) (GH-29122)
    1cdac61

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @methane methane

    Labels
    3.10 only security fixes 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @methane @nascheme