Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port importlib_resources (module and ABC) to Python 3.7 #76429

Closed
warsaw opened this issue Dec 7, 2017 · 19 comments
Closed

Port importlib_resources (module and ABC) to Python 3.7 #76429

warsaw opened this issue Dec 7, 2017 · 19 comments
Assignees
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@warsaw
Copy link
Member

warsaw commented Dec 7, 2017

BPO 32248
Nosy @Yhg1s, @warsaw, @brettcannon, @gpshead, @db3l, @vstinner, @ericsnowcurrently, @zware
PRs
  • bpo-32248: Implement importlib.abc.ResourceReader #4892
  • bpo-32248 - Implement importlib.resources #4911
  • bpo-32248: Introduce the concept of Loader.get_resource_reader() #5108
  • bpo-32248: install new test_importlib directories #5138
  • bpo-32248: Fix Git attributes of test_importlib data files #5209
  • bpo-32248: Fix test_importlib.test_open() #5213
  • bpo-32248 - Implement ResourceReader and get_resource_reader() for zipimport #5248
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/warsaw'
    closed_at = <Date 2018-01-24.20:38:34.046>
    created_at = <Date 2017-12-07.21:12:39.855>
    labels = ['3.7', 'type-feature', 'library']
    title = 'Port importlib_resources (module and ABC) to Python 3.7'
    updated_at = <Date 2018-01-24.20:38:34.044>
    user = 'https://github.com/warsaw'

    bugs.python.org fields:

    activity = <Date 2018-01-24.20:38:34.044>
    actor = 'barry'
    assignee = 'barry'
    closed = True
    closed_date = <Date 2018-01-24.20:38:34.046>
    closer = 'barry'
    components = ['Library (Lib)']
    creation = <Date 2017-12-07.21:12:39.855>
    creator = 'barry'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 32248
    keywords = ['patch']
    message_count = 19.0
    messages = ['307824', '308435', '308495', '308501', '308586', '309251', '309253', '309490', '309563', '309579', '309582', '309646', '309647', '309789', '309875', '310144', '310174', '310628', '310629']
    nosy_count = 8.0
    nosy_names = ['twouters', 'barry', 'brett.cannon', 'gregory.p.smith', 'db3l', 'vstinner', 'eric.snow', 'zach.ware']
    pr_nums = ['4892', '4911', '5108', '5138', '5209', '5213', '5248']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue32248'
    versions = ['Python 3.7']

    @warsaw
    Copy link
    Member Author

    warsaw commented Dec 7, 2017

    We intend to port importlib_resources to Python 3.7 as importlib.resources, with a provisional API. There's also a ResourceReader ABC to include, along with documentation and tests.

    Nosying Brett and assigning to myself, but if Brett *wants* to do the work, I won't stand in his way. :)

    @warsaw warsaw added the 3.7 (EOL) end of life label Dec 7, 2017
    @warsaw warsaw self-assigned this Dec 7, 2017
    @warsaw warsaw added the stdlib Python modules in the Lib dir label Dec 7, 2017
    @brettcannon
    Copy link
    Member

    New changeset 4ac5150 by Brett Cannon in branch 'master':
    bpo-32248: Implement importlib.abc.ResourceReader (GH-4892)
    4ac5150

    @warsaw
    Copy link
    Member Author

    warsaw commented Dec 17, 2017

    I have the tests and code ported, but there are still a few things to do, but here's a question:

    We're using type annotations in importlib_resources, which of course is the right decision. But I think we still have a moratorium on typing in the stdlib. So do we want to remove the types from importlib.resources?

    Doing so could make for more work as we continue to track importlib_resource, so I'm inclined not to remove them.

    @warsaw
    Copy link
    Member Author

    warsaw commented Dec 17, 2017

    PR 4911 implements the importlib.resources API, along with tests and documentation.

    @brett.cannon - I'm thinking we should do the native ResourceReader implementations for the built-in loaders as a separate branch.

    @brettcannon
    Copy link
    Member

    Yep, I assumed implementing the ReourceReader API would be a separate step.

    As for the type hints, I thought it was lifted such that new code could include it but we wouldn't be taking PRs to add them to pre-existing code?

    @warsaw
    Copy link
    Member Author

    warsaw commented Dec 30, 2017

    New changeset deae6b4 by Barry Warsaw in branch 'master':
    bpo-32248 - Implement importlib.resources (bpo-4911)
    deae6b4

    @warsaw
    Copy link
    Member Author

    warsaw commented Dec 30, 2017

    Next up - implementing the ResourceReader ABC.

    @db3l
    Copy link
    Contributor

    db3l commented Jan 5, 2018

    Apologies if this is obvious and already in progress, but the issue with Windows and this change appears to be that the "utf-8.file" file is being treated as text rather than binary.

    So the line ending is expanding to CRLF. The working copy of utf-8.file on my Windows workers is actually 21 bytes, not 20.

    @zware
    Copy link
    Member

    zware commented Jan 6, 2018

    I notice that my 'Installed' builder (http://buildbot.python.org/all/#/builders/103) has been broken since PR4911 landed. My suspicion is that it's just another directory missed in the Makefile install libinstall target, but haven't actually looked into it.

    @brettcannon
    Copy link
    Member

    I bet Barry forgot to add some test directories to the makefile.

    On Sat, Jan 6, 2018, 10:33 Zachary Ware, <report@bugs.python.org> wrote:

    Zachary Ware <zachary.ware@gmail.com> added the comment:

    I notice that my 'Installed' builder (
    http://buildbot.python.org/all/#/builders/103) has been broken since
    PR4911 landed. My suspicion is that it's just another directory missed in
    the Makefile install libinstall target, but haven't actually looked into it.

    ----------
    nosy: +zach.ware


    Python tracker <report@bugs.python.org>
    <https://bugs.python.org/issue32248\>


    @warsaw
    Copy link
    Member Author

    warsaw commented Jan 6, 2018

    On Jan 6, 2018, at 13:33, Zachary Ware <report@bugs.python.org> wrote:

    I notice that my 'Installed' builder (http://buildbot.python.org/all/#/builders/103) has been broken since PR4911 landed. My suspicion is that it's just another directory missed in the Makefile install libinstall target, but haven't actually looked into it.

    Dang. I’ll try to look into this as soon as possible, but it might be a day or so.

    @vstinner
    Copy link
    Member

    vstinner commented Jan 7, 2018

    Tests fail on Windows since the commit deae6b4.

    Example with AMD64 Windows8.1 Non-Debug 3.x:

    http://buildbot.python.org/all/#/builders/12/builds/426

    ======================================================================
    FAIL: test_open_binary (test.test_importlib.test_open.OpenDiskTests)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "D:\buildarea\3.x.ware-win81-release\build\lib\test\test_importlib\test_open.py", line 24, in test_open_binary
        self.assertEqual(result, b'Hello, UTF-8 world!\n')
    AssertionError: b'Hello, UTF-8 world!\r\n' != b'Hello, UTF-8 world!\n'

    @vstinner
    Copy link
    Member

    vstinner commented Jan 7, 2018

    Failure on x86 Gentoo Installed with X 3.x:

    http://buildbot.python.org/all/#/builders/103/builds/453

    4 tests failed:
    test_compileall test_import test_importlib test_inspect

    test test_import crashed -- Traceback (most recent call last):
      File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/libregrtest/runtest.py", line 163, in runtest_inner
        the_module = importlib.import_module(abstest)
      File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 994, in _gcd_import
      File "<frozen importlib._bootstrap>", line 971, in _find_and_load
      File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 723, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/test_import/__init__.py", line 28, in <module>
        from test.test_importlib.util import uncache
      File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/test_importlib/util.py", line 19, in <module>
        from . import data01
    ImportError: cannot import name 'data01' from 'test.test_importlib' (/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/test_importlib/__init__.py)

    @warsaw
    Copy link
    Member Author

    warsaw commented Jan 11, 2018

    @bbayles: I don't think PR 5149 is related to this issue.

    @brettcannon
    Copy link
    Member

    New changeset bca4218 by Brett Cannon in branch 'master':
    bpo-32248: Introduce the concept of Loader.get_resource_reader() (GH-5108)
    bca4218

    @vstinner
    Copy link
    Member

    New changeset 44a70e9 by Victor Stinner in branch 'master':
    Fix Git attributes of test_importlib data files (bpo-5209)
    44a70e9

    @vstinner
    Copy link
    Member

    New changeset 3a0cf93 by Victor Stinner in branch 'master':
    bpo-32248: Fix test_importlib.test_open() (bpo-5213)
    3a0cf93

    @warsaw
    Copy link
    Member Author

    warsaw commented Jan 24, 2018

    New changeset 6f6eb35 by Barry Warsaw in branch 'master':
    bpo-32248 - Implement ResourceReader and get_resource_reader() for zipimport (bpo-5248)
    6f6eb35

    @warsaw
    Copy link
    Member Author

    warsaw commented Jan 24, 2018

    Now that the zipimporter support has landed, I think we're done!

    @warsaw warsaw closed this as completed Jan 24, 2018
    @warsaw warsaw added the type-feature A feature request or enhancement label Jan 24, 2018
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants