Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

zipimport raises NameError: name '_boostrap_external' is not defined #83214

Closed
misho88 mannequin opened this issue Dec 12, 2019 · 11 comments
Closed

zipimport raises NameError: name '_boostrap_external' is not defined #83214

misho88 mannequin opened this issue Dec 12, 2019 · 11 comments
Labels
3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@misho88
Copy link
Mannequin

misho88 mannequin commented Dec 12, 2019

BPO 39033
Nosy @Yhg1s, @ncoghlan, @encukou, @zware, @serhiy-storchaka, @animalize, @miss-islington, @tirkarthi, @misho88
PRs
  • bpo-39033: Fix NameError in zipimport during hash validation #17588
  • [3.8] bpo-39033: Fix NameError in zipimport during hash validation (GH-17588) #17642
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-01-14.11:41:34.654>
created_at = <Date 2019-12-12.22:20:17.746>
labels = ['3.8', 'library', '3.9', 'type-crash']
title = "zipimport raises NameError: name '_boostrap_external' is not defined"
updated_at = <Date 2020-01-14.11:41:34.650>
user = 'https://github.com/misho88'

    bugs.python.org fields:

    activity = <Date 2020-01-14.11:41:34.650>
actor = 'petr.viktorin'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-14.11:41:34.654>
closer = 'petr.viktorin'
components = ['Library (Lib)']
creation = <Date 2019-12-12.22:20:17.746>
creator = 'misho88'
dependencies = []
files = []
hgrepos = []
issue_num = 39033
keywords = ['patch']
message_count = 11.0
messages = ['358310', '358311', '358316', '358317', '358318', '358319', '358321', '358326', '358459', '359964', '359965']
nosy_count = 9.0
nosy_names = ['twouters', 'ncoghlan', 'petr.viktorin', 'zach.ware', 'serhiy.storchaka', 'malin', 'miss-islington', 'xtreak', 'misho88']
pr_nums = ['17588', '17642']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue39033'
versions = ['Python 3.8', 'Python 3.9']
    @misho88
    Copy link
    Mannequin Author

    misho88 mannequin commented Dec 12, 2019

    I think there's a "t" missing:

    Lib/zipimport.py
    609-
    610- try:
    611: _boostrap_external._validate_hash_pyc(
    612- data, source_hash, fullname, exc_details)
    613- except ImportError:
    614- return None
    615- else:
    616- source_mtime, source_size = \
    617- _get_mtime_and_size_of_source(self, fullpath)
    618-
    619- if source_mtime:

    @misho88 misho88 mannequin added 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump labels Dec 12, 2019
    @zware
    Copy link
    Member

    zware commented Dec 12, 2019

    Good catch! Would you like to submit a pull request to fix it? Ideally such a PR should also include a test to exercise this code.

    @animalize
    Copy link
    Mannequin

    animalize mannequin commented Dec 13, 2019

    Is it possible to scan stdlib to find similar bugs?

    @tirkarthi
    Copy link
    Member

    A possible test case to trigger this code path based on testUncheckedHashBasedPyc would be as below. check_hash_based_pycs can be patched to be "always" so that the hash is validated using _bootstrap_external._validate_hash_pyc. The difference between "state = old" for pyc and "state = new" in py would ensure the hashes are different to raise ImportError expected and the test picks up state = new as the updated source code. Without fixing the typo trying to make hash validation would throw NameError.

    @unittest.mock.patch('_imp.check_hash_based_pycs', 'always')
    def test_checked_hash_based_change_pyc(self):
    source = b"state = 'old'"
    source_hash = importlib.util.source_hash(source)
    bytecode = importlib._bootstrap_external._code_to_hash_pyc(
    compile(source, "???", "exec"),
    source_hash,
    False,
    )
    files = {TESTMOD + ".py": (NOW, "state = 'new'"),
    TESTMOD + ".pyc": (NOW - 20, bytecode)}
    def check(mod):
    self.assertEqual(mod.state, 'new')
    self.doTest(None, files, TESTMOD, call=check)

    @tirkarthi
    Copy link
    Member

    Also since the fix involves modifying zipimport.py it would also require running "make regen-importlib && make -s" for changes to ensure the updated regen code is used.

    @tirkarthi
    Copy link
    Member

    Ma Lin, Running pylint/flake8 could possibly detect it but needs to be manually checked. bpo-36948 was a similar report I filed in the past using flake8 regarding NameError.

    pylint - Lib/zipimport.py:611:20: E0602: Undefined variable '_boostrap_external' (undefined-variable)
    flake8 - Lib/zipimport.py:611:21: F821 undefined name '_boostrap_external'

    @serhiy-storchaka
    Copy link
    Member

    Wince you has virtually written a patch, do you mind to create a PR Karthikeyan?

    @tirkarthi
    Copy link
    Member

    Sure Serhiy, I will create one.

    @ncoghlan
    Copy link
    Contributor

    New changeset 79f02fe by Nick Coghlan (Xtreak) in branch 'master':
    bpo-39033: Fix NameError in zipimport during hash validation (GH-17588)
    79f02fe

    @miss-islington
    Copy link
    Contributor

    New changeset 9955f33 by Miss Islington (bot) (Karthikeyan Singaravelan) in branch '3.8':
    [3.8] bpo-39033: Fix NameError in zipimport during hash validation (GH-17588) (GH-17642)
    9955f33

    @encukou
    Copy link
    Member

    encukou commented Jan 14, 2020

    Thank you, Mihail and Karthikeyan!

    @encukou encukou closed this as completed Jan 14, 2020
    @encukou encukou closed this as completed Jan 14, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    6 participants
    @encukou @ncoghlan @serhiy-storchaka @tirkarthi @zware @miss-islington