Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

checked hash-based pyc files not working with imp module #78237

Closed
phmccarty mannequin opened this issue Jul 5, 2018 · 6 comments
Closed

checked hash-based pyc files not working with imp module #78237

phmccarty mannequin opened this issue Jul 5, 2018 · 6 comments
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@phmccarty
Copy link
Mannequin

phmccarty mannequin commented Jul 5, 2018

BPO 34056
Nosy @brettcannon, @benjaminp, @miss-islington, @phmccarty
PRs
  • bpo-34056: Always return bytes from _HackedGetData.get_data(). #8130
  • [3.7] closes bpo-34056: Always return bytes from _HackedGetData.get_data(). (GH-8130) #8146
    		•
    Files
  • imp-test.py: imp test code
  • imp-test-mod.py: imp test module
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-07-07.03:41:11.077>
created_at = <Date 2018-07-05.22:31:12.837>
labels = ['interpreter-core', 'type-bug', '3.7']
title = 'checked hash-based pyc files not working with imp module'
updated_at = <Date 2018-07-07.04:00:48.180>
user = 'https://github.com/phmccarty'

    bugs.python.org fields:

    activity = <Date 2018-07-07.04:00:48.180>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2018-07-07.03:41:11.077>
closer = 'benjamin.peterson'
components = ['Interpreter Core']
creation = <Date 2018-07-05.22:31:12.837>
creator = 'phmccarty'
dependencies = []
files = ['47671', '47672']
hgrepos = []
issue_num = 34056
keywords = ['patch']
message_count = 6.0
messages = ['321139', '321147', '321149', '321150', '321202', '321203']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'benjamin.peterson', 'miss-islington', 'phmccarty']
pr_nums = ['8130', '8146']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue34056'
versions = ['Python 3.7']
    @phmccarty
    Copy link
    Mannequin Author

    phmccarty mannequin commented Jul 5, 2018

    OS: Clear Linux build 23460
    Python version: 3.7.0

    Description:
    I am seeing an uncaught exception in Python 3.7.0 when using the "imp" module to import a module that has a checked hash-based pyc file. See the attached source files.

    Steps to reproduce:

    1. Copy attached source files to a directory
    2. In that directory, run
      $ python3.7 -m compileall --invalidation-mode checked-hash imp-test-mod.py
      $ python3.7 imp-test.py
    3. See the resulting output (omitting the imp deprecation notice):
    Traceback (most recent call last):
  File "imp-test.py", line 6, in <module>
    mod = imp.load_module(modname, f, p, d)
  File "/usr/lib/python3.7/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.7/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 724, in exec_module
  File "<frozen importlib._bootstrap_external>", line 838, in get_code
TypeError: a bytes-like object is required, not 'str'

    @phmccarty phmccarty mannequin added 3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Jul 5, 2018
    @benjaminp
    Copy link
    Contributor

    Can you not use imp? It's been deprecated since 3.4 and gradually become more and more broken and useless as this and other import improvements (e.g., namespace packages) have happened.

    @phmccarty
    Copy link
    Mannequin Author

    phmccarty mannequin commented Jul 6, 2018

    Thanks for the response.

    I would like to avoid using imp, but I work on a distro team that ships many packages that still use the module. We wanted to start using checked-hash invalidation right away after upgrading to 3.7.0, but some of our release tests failed due to this issue.

    I am willing to file bugs against other projects still using imp to stop using it, if that is the recommended direction. Considering that imp still works with timestamp invalidation, and checked-hash invalidation being a brand new feature, I wanted to file a bug here first before reaching out to the other projects.

    @benjaminp
    Copy link
    Contributor

    I see. Thanks for being an early adopter! We can probably hack something together for imp. I'll send a PR in a moment.

    Filing bugs about imp usage would be valuable, especially if it flushes out any cases where imp provides functionality that our newer APIs don't. We'd quite like to kill imp.

    @benjaminp
    Copy link
    Contributor

    New changeset b0274f2 by Benjamin Peterson in branch 'master':
    closes bpo-34056: Always return bytes from _HackedGetData.get_data(). (GH-8130)
    b0274f2

    @miss-islington
    Copy link
    Contributor

    New changeset 7bd6f0e by Miss Islington (bot) in branch '3.7':
    closes bpo-34056: Always return bytes from _HackedGetData.get_data(). (GH-8130)
    7bd6f0e

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @benjaminp @miss-islington