classification
Title: Port importlib_resources (module and ABC) to Python 3.7
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: barry Nosy List: barry, brett.cannon, db3l, eric.snow, gregory.p.smith, twouters, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2017-12-07 21:12 by barry, last changed 2018-01-24 20:38 by barry. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4892 closed brett.cannon, 2017-12-15 18:42
PR 4911 merged barry, 2017-12-17 23:16
PR 5108 merged brett.cannon, 2018-01-05 23:32
PR 5138 merged ned.deily, 2018-01-09 01:47
PR 5209 merged vstinner, 2018-01-16 22:38
PR 5213 merged vstinner, 2018-01-17 10:37
PR 5248 merged barry, 2018-01-20 00:35
Messages (19)
msg307824 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-12-07 21:12
We intend to port importlib_resources to Python 3.7 as importlib.resources, with a provisional API.  There's also a ResourceReader ABC to include, along with documentation and tests.

Nosying Brett and assigning to myself, but if Brett *wants* to do the work, I won't stand in his way. :)
msg308435 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-12-16 00:29
New changeset 4ac5150e068a3a795ef00465f6dff51747b62b91 by Brett Cannon in branch 'master':
bpo-32248: Implement importlib.abc.ResourceReader (GH-4892)
https://github.com/python/cpython/commit/4ac5150e068a3a795ef00465f6dff51747b62b91
msg308495 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-12-17 19:18
I have the tests and code ported, but there are still a few things to do, but here's a question:

We're using type annotations in importlib_resources, which of course is the right decision.  But I think we still have a moratorium on typing in the stdlib.  So do we want to remove the types from importlib.resources?

Doing so could make for more work as we continue to track importlib_resource, so I'm inclined not to remove them.
msg308501 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-12-17 23:18
PR 4911 implements the importlib.resources API, along with tests and documentation.

@brett.cannon - I'm thinking we should do the native ResourceReader implementations for the built-in loaders as a separate branch.
msg308586 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-12-18 21:59
Yep, I assumed implementing the ReourceReader API would be a separate step.

As for the type hints, I thought it was lifted such that new code could include it but we wouldn't be taking PRs to add them to pre-existing code?
msg309251 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-12-30 20:18
New changeset deae6b451fefd5fd3143dd65051e1d341e5a5f84 by Barry Warsaw in branch 'master':
bpo-32248 - Implement importlib.resources (#4911)
https://github.com/python/cpython/commit/deae6b451fefd5fd3143dd65051e1d341e5a5f84
msg309253 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-12-30 20:20
Next up - implementing the ResourceReader ABC.
msg309490 - (view) Author: David Bolen (db3l) Date: 2018-01-05 01:06
Apologies if this is obvious and already in progress, but the issue with Windows and this change appears to be that the "utf-8.file" file is being treated as text rather than binary.

So the line ending is expanding to CRLF.  The working copy of utf-8.file on my Windows workers is actually 21 bytes, not 20.
msg309563 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2018-01-06 18:33
I notice that my 'Installed' builder (http://buildbot.python.org/all/#/builders/103) has been broken since PR4911 landed.  My suspicion is that it's just another directory missed in the Makefile install libinstall target, but haven't actually looked into it.
msg309579 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-01-06 20:56
I bet Barry forgot to add some test directories to the makefile.

On Sat, Jan 6, 2018, 10:33 Zachary Ware, <report@bugs.python.org> wrote:

>
> Zachary Ware <zachary.ware@gmail.com> added the comment:
>
> I notice that my 'Installed' builder (
> http://buildbot.python.org/all/#/builders/103) has been broken since
> PR4911 landed.  My suspicion is that it's just another directory missed in
> the Makefile install libinstall target, but haven't actually looked into it.
>
> ----------
> nosy: +zach.ware
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue32248>
> _______________________________________
>
msg309582 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-01-06 21:30
On Jan 6, 2018, at 13:33, Zachary Ware <report@bugs.python.org> wrote:
> 
> I notice that my 'Installed' builder (http://buildbot.python.org/all/#/builders/103) has been broken since PR4911 landed.  My suspicion is that it's just another directory missed in the Makefile install libinstall target, but haven't actually looked into it.

Dang.  I’ll try to look into this as soon as possible, but it might be a day or so.
msg309646 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-01-07 22:47
Tests fail on Windows since the commit deae6b451fefd5fd3143dd65051e1d341e5a5f84.

Example with AMD64 Windows8.1 Non-Debug 3.x:

http://buildbot.python.org/all/#/builders/12/builds/426

======================================================================
FAIL: test_open_binary (test.test_importlib.test_open.OpenDiskTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\buildarea\3.x.ware-win81-release\build\lib\test\test_importlib\test_open.py", line 24, in test_open_binary
    self.assertEqual(result, b'Hello, UTF-8 world!\n')
AssertionError: b'Hello, UTF-8 world!\r\n' != b'Hello, UTF-8 world!\n'
msg309647 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-01-07 22:50
Failure on x86 Gentoo Installed with X 3.x:

http://buildbot.python.org/all/#/builders/103/builds/453

4 tests failed:
    test_compileall test_import test_importlib test_inspect

test test_import crashed -- Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/libregrtest/runtest.py", line 163, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 723, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/test_import/__init__.py", line 28, in <module>
    from test.test_importlib.util import uncache
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/test_importlib/util.py", line 19, in <module>
    from . import data01
ImportError: cannot import name 'data01' from 'test.test_importlib' (/buildbot/buildarea/3.x.ware-gentoo-x86.installed/build/target/lib/python3.7/test/test_importlib/__init__.py)
msg309789 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-01-11 04:30
@bbayles: I don't think PR 5149 is related to this issue.
msg309875 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-01-12 23:09
New changeset bca42186b69e2e615d29d0d4fdb493c9fe71c48b by Brett Cannon in branch 'master':
bpo-32248: Introduce the concept of Loader.get_resource_reader() (GH-5108)
https://github.com/python/cpython/commit/bca42186b69e2e615d29d0d4fdb493c9fe71c48b
msg310144 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-01-17 09:26
New changeset 44a70e9336765404057b12345465844dc291297a by Victor Stinner in branch 'master':
Fix Git attributes of test_importlib data files (#5209)
https://github.com/python/cpython/commit/44a70e9336765404057b12345465844dc291297a
msg310174 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-01-17 14:21
New changeset 3a0cf931796b9ad1b4833ab89c79d284b4253c91 by Victor Stinner in branch 'master':
bpo-32248: Fix test_importlib.test_open() (#5213)
https://github.com/python/cpython/commit/3a0cf931796b9ad1b4833ab89c79d284b4253c91
msg310628 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-01-24 20:36
New changeset 6f6eb35f9bee18f54945f09664344f2d118ed89f by Barry Warsaw in branch 'master':
 bpo-32248 - Implement `ResourceReader` and `get_resource_reader()` for zipimport (#5248)
https://github.com/python/cpython/commit/6f6eb35f9bee18f54945f09664344f2d118ed89f
msg310629 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-01-24 20:38
Now that the zipimporter support has landed, I think we're done!
History
Date User Action Args
2018-01-24 20:38:34barrysetstatus: open -> closed
type: enhancement
messages: + msg310629

resolution: fixed
stage: patch review -> resolved
2018-01-24 20:36:24barrysetmessages: + msg310628
2018-01-20 00:35:15barrysetpull_requests: + pull_request5093
2018-01-17 14:21:55vstinnersetmessages: + msg310174
2018-01-17 10:37:00vstinnersetpull_requests: + pull_request5067
2018-01-17 09:26:11vstinnersetmessages: + msg310144
2018-01-16 22:58:34barrysetpull_requests: - pull_request5016
2018-01-16 22:38:25vstinnersetpull_requests: + pull_request5064
2018-01-12 23:09:02brett.cannonsetmessages: + msg309875
2018-01-11 13:51:15bbaylessetpull_requests: + pull_request5016
2018-01-11 04:30:52barrysetpull_requests: - pull_request5006
2018-01-11 04:30:44barrysetmessages: + msg309789
2018-01-11 00:33:03bbaylessetpull_requests: + pull_request5006
2018-01-09 01:47:56ned.deilysetpull_requests: + pull_request4998
2018-01-07 22:50:16vstinnersetmessages: + msg309647
2018-01-07 22:47:58vstinnersetnosy: + vstinner
messages: + msg309646
2018-01-06 21:30:03barrysetmessages: + msg309582
2018-01-06 20:56:51brett.cannonsetmessages: + msg309579
2018-01-06 18:33:29zach.waresetnosy: + zach.ware
messages: + msg309563
2018-01-05 23:32:52brett.cannonsetpull_requests: + pull_request4975
2018-01-05 01:06:47db3lsetnosy: + db3l
messages: + msg309490
2017-12-30 20:20:49barrysetmessages: + msg309253
2017-12-30 20:18:08barrysetmessages: + msg309251
2017-12-20 00:43:30eric.snowsetnosy: + eric.snow
2017-12-18 21:59:47brett.cannonsetmessages: + msg308586
2017-12-17 23:18:45barrysetmessages: + msg308501
2017-12-17 23:16:09barrysetpull_requests: + pull_request4805
2017-12-17 19:18:22barrysetmessages: + msg308495
2017-12-16 00:29:41brett.cannonsetmessages: + msg308435
2017-12-15 18:42:17brett.cannonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request4785
2017-12-08 19:40:42gregory.p.smithsetnosy: + twouters, gregory.p.smith
2017-12-07 21:12:39barrycreate