New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Loader for namespace packages #79854
Comments
The documentation for import lib.machinery.ModuleSpec says that the attribute "loader" should be None for namespace packages (see <https://docs.python.org/3/library/importlib.html#importlib.machinery.ModuleSpec\>) In reality the loader for namespace packages is an instance of a private class, and that class does not conform to the importlib.abc.Loader ABC. To reproduce:
Python 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import namespace
>>> namespace.__loader__
<_frozen_importlib_external._NamespaceLoader object at 0x104c7bdd8>
>>> import importlib.abc
>>> isinstance(namespace.__loader__, importlib.abc.Loader)
False
>>> import importlib.util
>>> importlib.util.find_spec('namespace')
ModuleSpec(name='namespace', loader=<_frozen_importlib_external._NamespaceLoader object at 0x104c7bdd8>, submodule_search_locations=_NamespacePath(['/Users/ronald/Projects/pyobjc-hg/modulegraph2/namespace'])) Note how "namespace" has an attribute named "__loader__" that is not None, and the same is true for the ModuleSpec found using importlib.util.find_spec. The loader does not claim to conform to any Loader ABC (but provides all methods required for conformance to the InspectLoader ABC) I'm not sure if this should be two issues:
P.S. the latter is also true for zipimport.zipimporter. |
On the first point, I'd categorize this as a documentation bug, and in fact, it's inconsistent with the language reference, which doesn't have the same language: https://docs.python.org/3/reference/import.html#__loader__ On the second point, it probably does make sense to register the ABC. |
@barry: I agree on both. Do you know why the namespace package loader lies about the source and code? Both .get_source() and .get_code() return a value that isn't None. And likewise: Why is the namespace package loader a private class, other loaders are exposed in importlib.machinery? This makes it hard to detect PEP-420 style namespace packages without relying on private APIs, esp. combined with the behaviour of .get_source() and .get_code(). |
On Jan 7, 2019, at 03:16, Ronald Oussoren <report@bugs.python.org> wrote:
I don’t remember the history of this. I wonder if Brett or Eric have any additional insights. |
Namespace packages (PEP-420) predate ModuleSpec (PEP-451). So, I think this probably happened when 451 was implemented. Maybe Eric Snow recalls? I say this without having looked at it very deeply. As to why the namespace package loader is a private class: it never occurred to me someone would care about inspecting it. |
The background for all of this: I'm currently rewriting modulegraph (https://pypi.org/project/modulegraph/) to use importlib instead of its own implementation of the import mechanism. I currently detect PEP-420 style namespace packages, but I'm not sure if I really need that information. With the current behaviour of the namespace loader I probably do, because I'd otherwise end up with creating an __init__.py for these packages when I use the generated module graph in py2app to copy modules into an application bundle. |
I think exposing _NamespaceLoader as NamespaceLoader and registering the ABC make sense. That would make this in to a feature request for 3.8. |
Not sure if it's proper etiquette to bump issues on the tracker, but is there any interest in this issue for 3.11? |
I think a PR with tests would be a good first step. |
I'm going to take a look at this during the Python core sprint. |
First crack at a PR for this issue. |
Since the documentation problem reported here has since been fixed, and really all that's left is to expose NamespaceLoader publicly and register it with the abc, this is technically a new feature so it can't be backported. Thus, targeting only 3.11. |
Should we register with the ABC or is it time to do proper typing.Protocol classes and have the ABCs inherit from those? |
I don't know. What benefit would be gained? That should probably be a separate issue/PR in either case. |
The ABCs are broader than what the import system actually requires due to their helper methods. So for typing purposes they are actually not a perfect fit.
https://bugs.python.org/issue38782 and I was trying to rope you into doing the work. 😁 |
On Mon, Jan 7, 2019 at 11:41 PM Eric V. Smith <report@bugs.python.org> wrote:
PEP-451 talks about this a little """ |
On Oct 20, 2021, at 11:27, Brett Cannon <report@bugs.python.org> wrote:
Ha! You should have nosied me then :D - but anyway I’m following that ticket now so we’ll see. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: