This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [doc] Clarify that zipimport does not invoke import hooks to load custom files from zip.
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Decorater, brett.cannon, docs@python, eric.snow, iritkatriel, miss-islington, ncoghlan, twouters
Priority: normal Keywords: easy, patch

Created on 2016-11-28 09:45 by Decorater, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
zipimport.rst.patch Decorater, 2016-12-30 09:43 review
Pull Requests
URL Status Linked Edit
PR 30060 merged iritkatriel, 2021-12-11 17:41
PR 30133 merged miss-islington, 2021-12-16 09:12
PR 30134 merged miss-islington, 2021-12-16 09:12
Messages (8)
msg281850 - (view) Author: Decorater (Decorater) * Date: 2016-11-28 09:45
I am wondering so lets say for example if I was to make a json import hook (code below):

from importlib.machinery import FileFinder
import json
import sys


class ExtensionImporter(object):
    """Base Importer Class."""
    def __init__(self, extension_list):
        self.extensions = extension_list

    def find_loader(self, name, path):
        """Filds some loaders for importing the module."""
        self.path = path
        return self

    def load_module(self, fullname):
        if fullname in sys.modules:
            return sys.modules[fullname]
        return None


class JsonImporter(ExtensionImporter):
    """JSON importer Class."""
    def __init__(self):
        super(JsonImporter, self).__init__(['.json'])

    def load_module(self, fullname):
        """Loads modules found with the extension"""
        premodule = super(JsonImporter, self).load_module(fullname)
        if premodule is None:
             with open(self.path, 'r') as f:
                module = json.load(f)
                sys.modules[fullname] = module
                return module
            raise ImportError('Couldn't open path')

extension_importers = [JsonImporter()]
hook_list = []
for importer in extension_importers:
    hook_list.append((importer.find_loader, importer.extensions))

sys.path_hooks.insert(0, FileFinder.path_hook(*hook_list))
sys.path_importer_cache.clear()


What if I had those json files in a zip file and used ZipImport on that zip file would it use that import hook that I shared above if all other import methods fail to import those json files (obvously they would fail though)?

There should be documentation to explain if it supports user created import hooks that they create using importlib.

This is because I would be very happy if zipimport supports my own import hook if any other method of importing files from that zip file fails but mine ends up working. It also allows people to load up many other formats (provided they know the format) as if it was a python script. (of if they compressed it in a custom file like for example a RAR file that WinRAR can create. And yes I would support a RARImport for rar files.
msg283446 - (view) Author: Decorater (Decorater) * Date: 2016-12-16 22:48
Well, Will there be any documentations for this before 3.6 final release or no because this would be nice to know for other people who would like to do something similar.
msg283505 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-12-17 19:13
The cutoff for 3.6.0 has passed, but documentation patches can still go in for 3.6.1 (and 3.5.3).
msg284325 - (view) Author: Decorater (Decorater) * Date: 2016-12-30 09:43
OK, Well I just tested and it sadly don't support import hooks that adds support for importing custom file types or file types python does know about with a uncommon extension inside of zip files which is somewhat sad. However if someone was to do something similar to py2exe's import hook allowing memory loading pyd and so files they could allow it to work for their import hook's needs.
msg407716 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-05 16:10
I think that
(1) the patch is not worded very clearly
(2) the docs should not reference py2exe for instructions how to do something

Perhaps it would suffice to change the sentence:

Any files may be present in the ZIP archive, but only files .py and .pyc are available for import

to:

Any files may be present in the ZIP archive, but the importer is only invoked for .py and .pyc files

?
msg408677 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-16 09:12
New changeset a951c95a13c3555ac8fb1c8ee615ba3930ccc6f7 by Irit Katriel in branch 'main':
bpo-28816: [doc] clarify that zipimport invokes importers only for python files (GH-30060)
https://github.com/python/cpython/commit/a951c95a13c3555ac8fb1c8ee615ba3930ccc6f7
msg408697 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-16 12:23
New changeset 4c1effaaee472cc67f3186411a3df4f39af3d71a by Miss Islington (bot) in branch '3.9':
bpo-28816: [doc] clarify that zipimport invokes importers only for python files (GH-30060) (GH-30134)
https://github.com/python/cpython/commit/4c1effaaee472cc67f3186411a3df4f39af3d71a
msg408699 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-16 12:23
New changeset 0194bbbee6a12264e93d3217c774e226f0a1737d by Miss Islington (bot) in branch '3.10':
bpo-28816: [doc] clarify that zipimport invokes importers only for python files (GH-30060) (GH-30133)
https://github.com/python/cpython/commit/0194bbbee6a12264e93d3217c774e226f0a1737d
History
Date User Action Args
2022-04-11 14:58:40adminsetgithub: 73002
2021-12-16 12:24:31iritkatrielsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-12-16 12:23:45iritkatrielsetmessages: + msg408699
2021-12-16 12:23:29iritkatrielsetmessages: + msg408697
2021-12-16 09:12:48miss-islingtonsetpull_requests: + pull_request28354
2021-12-16 09:12:43miss-islingtonsetnosy: + miss-islington

pull_requests: + pull_request28353
stage: patch review
2021-12-16 09:12:31iritkatrielsetmessages: + msg408677
2021-12-11 17:41:42iritkatrielsetstatus: pending -> open
pull_requests: + pull_request28280
2021-12-05 16:10:36iritkatrielsetstatus: open -> pending

type: enhancement

title: Document if zipimport can respect import hooks to load custom files from zip. -> [doc] Clarify that zipimport does not invoke import hooks to load custom files from zip.
keywords: + easy
nosy: + iritkatriel
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.4, Python 3.5, Python 3.6, Python 3.7
messages: + msg407716
2016-12-30 09:43:41Decoratersetfiles: + zipimport.rst.patch
keywords: + patch
messages: + msg284325
2016-12-17 19:13:03brett.cannonsetmessages: + msg283505
2016-12-16 22:48:26Decoratersetmessages: + msg283446
2016-11-29 00:35:32Decoratersetnosy: + brett.cannon, ncoghlan, eric.snow
2016-11-28 21:38:02Decoratersetnosy: + twouters
2016-11-28 21:35:42Decoratersetnosy: - twouters, brett.cannon, paul.moore, ncoghlan, tim.golden, eric.snow, zach.ware
2016-11-28 14:40:13steve.dowersetnosy: + twouters, brett.cannon, ncoghlan, eric.snow, - steve.dower
2016-11-28 13:31:27Decoratersetcomponents: - Windows
2016-11-28 09:46:38Decoratersetassignee: docs@python

components: + Documentation, - Interpreter Core
nosy: + docs@python
2016-11-28 09:45:40Decoratercreate