Title: importlib.find_spec raises AttributeError when parent is not a package/module
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, eric.snow, ncoghlan, tkhyn, zvyn
Priority: normal Keywords:

Created on 2017-05-23 00:47 by tkhyn, last changed 2017-06-14 21:35 by brett.cannon. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 1899 merged zvyn, 2017-06-01 03:14
Messages (6)
msg294209 - (view) Author: (tkhyn) Date: 2017-05-23 00:47
Hello, I stumbled upon this issue when using the module_has_submodule function in Django, which raised an exception when trying to import a dotted path such as ``parent.module`` when ``parent`` does not exist or is not a package. I would expect (as well as the django devs, apparently) find_spec to return None instead of raising an AttributeError or ModuleNotFoundError. 

Unless you think Django or any package making use of importlib.find_spec should handle these exceptions, the fix is quite simple.

Steps to reproduce (with Python 3.6.1):

>>> from importlib.util import find_spec
>>> find_spec('parent.module')
  File "C:\Python\3.6\Lib\importlib\", line 89, in find_spec
    return _find_spec(fullname, parent.__path__)
AttributeError: module 'parent' has no attribute '__path__'
>>> find_spec('invalid_parent.module')
  File "C:\Python\3.6\Lib\importlib\", line 88, in find_spec
    parent = __import__(parent_name, fromlist=['__path__'])
ModuleNotFoundError: No module named 'invalid_parent'

The fix is quite simple, replacing

    if fullname not in sys.modules:
        parent_name = fullname.rpartition('.')[0]
        if parent_name:
            # Use builtins.__import__() in case someone replaced it.

            parent = __import__(parent_name, fromlist=['__path__'])
            return _find_spec(fullname, parent.__path__)


            return _find_spec(fullname, None)

    if fullname not in sys.modules:
        parent_name = fullname.rpartition('.')[0]
        if parent_name:
            # Use builtins.__import__() in case someone replaced it.
                parent = __import__(parent_name, fromlist=['__path__']).__path__

            except (AttributeError, ModuleNotFoundError):
                # parent is not a package
                return None
            parent = None
        return _find_spec(fullname, parent)

in importlib.util.find_spec.
msg294280 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-05-23 19:48
The key thing to think about is do you think find_spec("parent.module") is working with a single thing called "parent.module" or is it working with two separate things of "parent" and "module" which happens to be contained on "parent"? If you take the former view then you get the current semantics, but if you view it as the latter then you get the semantics you're suggesting, tkhyn.

My inclination is for the former semantics (i.e. think of it as a really long name for a specific module where it turns out the name is broken). If you look at it as find_spec(".submodule", package="parent") this also visually supports the idea that parent modules shouldn't trigger a None return. Finally, this would break any code that expects the current semantics.

So thanks for the bug report, but I'm going to close this as "not a bug".
msg294281 - (view) Author: (tkhyn) Date: 2017-05-23 20:07
Ok, thanks for the reply. Actually the thing that bothered me was the AttributeError exception. I would probably not have opened a ticket should find_spec have raised a ModuleNotFoundError (in line with import_module).

Would you consider catching the AttributeError (which means detecting if parent_name relates to a package) to raise a ModuleNotFoundError instead more appropriate?
msg294383 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-05-24 19:41
Here is why Python does when importing a module that lacks __path__:

>>> import importlib
>>> del importlib.__path__
>>> import importlib.util
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'importlib.util'; 'importlib' is not a package

So yes, we should change find_spec() to raise ModuleNotFoundError to match (although only in Python 3.7 since this is a breaking change).
msg294903 - (view) Author: Milan Oberkirch (zvyn) * Date: 2017-06-01 03:18
I added a PR changing the exception raised as suggested, reviews welcome!
msg296041 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2017-06-14 21:34
New changeset 8c3f05e9f0f0b30a3d4a2433e92471794d8258af by Brett Cannon (Milan Oberkirch) in branch 'master':
bpo-30436: Raise ModuleNotFoundError for importlib.util.find_spec() when parent isn't a package (GH-1899)
Date User Action Args
2017-06-14 21:35:16brett.cannonsetstatus: open -> closed
resolution: fixed
stage: resolved
2017-06-14 21:34:52brett.cannonsetmessages: + msg296041
2017-06-01 03:18:22zvynsetnosy: + zvyn
messages: + msg294903
2017-06-01 03:15:00zvynsetpull_requests: + pull_request1977
2017-05-24 19:41:17brett.cannonsetstatus: closed -> open
versions: + Python 3.7, - Python 3.6
title: importlib.find_spec raises AttributeError/ModuleNotFoundError when parent is not a package/module -> importlib.find_spec raises AttributeError when parent is not a package/module
messages: + msg294383

resolution: not a bug -> (no value)
stage: resolved -> (no value)
2017-05-23 20:07:37tkhynsetmessages: + msg294281
2017-05-23 19:48:18brett.cannonsetstatus: open -> closed

type: behavior

nosy: + brett.cannon, ncoghlan, eric.snow
messages: + msg294280
resolution: not a bug
stage: resolved
2017-05-23 00:47:04tkhyncreate