classification
Title: docs claim __import__ checked for in globals, but IMPORT_NAME bytecode does not
Type: Stage: resolved
Components: Documentation, Library (Lib) Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: brett.cannon, docs@python, eric.snow, python-dev, r.david.murray, superbobry
Priority: normal Keywords:

Created on 2015-10-28 21:34 by superbobry, last changed 2015-12-04 22:54 by brett.cannon. This issue is now closed.

Messages (5)
msg253635 - (view) Author: Sergei Lebedev (superbobry) Date: 2015-10-28 21:34
According to the current import system documentation

> When calling ``__import__()`` as part of an import statement, the import system first checks the module global namespace for a function by that name. If it is not found, then the standard builtin ``__import__()`` is called.

However, one can easily verify this isn't (always) the case::

    import sys

    assert "glob" not in sys.modules
    __import__ = print
   
    import glob  # Doesn't print anything.

I've traced the import statement from ``ceval.c`` to the frozen ``importlib._bootstrap`` and it seems the cause of the problem is in ``_find_and_load_unlocked``, which simply ignores the ``_import`` argument in the case above::

    def _find_and_load_unlocked(name, import_):
        path = None
        # ... parent processing ...
        spec = _find_spec(name, path)
        if spec is None:
            raise ImportError(_ERR_MSG.format(name), name=name)
        else:
            # XXX import_ is not used.
            module = _SpecMethods(spec)._load_unlocked()
        # ... more parent processing ...
        return module

I'm not sure if this is a bug in the documentation or implementation, so any feedback is appreciated.
msg253638 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-10-28 22:41
I think the documentation is wrong. Going all the way back to Python 2.7, you will see that importing a module emits IMPORT_NAME as the bytecode. That bytecode only looks for __import__ in the builtin  namespace: https://hg.python.org/cpython/file/2.7/Python/ceval.c#l2588 . That then calls PyImport_ImportModuleLevel(): https://hg.python.org/cpython/file/2.7/Python/bltinmodule.c#l49 . That then just ends up executing import.

If you look at PyImport_Import() in Python 2.7 that comes the closest to what the docs reference, but that still uses __builtins__.__import__ based on finding __builtins__ from the global namespace: https://hg.python.org/cpython/file/2.7/Python/import.c#l2825 . But that isn't directly called by import itself and is just a C-level API.

If you look in Python 3, then you will see that as far back as Python 3.3 the code in importlib is more or less the same: __import__ is used when importing a parent and then importlib is used for the rest: https://hg.python.org/cpython/file/default/Lib/importlib/_bootstrap.py#l944 . This was introduced so that the accelerated C version of __import__ would be used to import the parent rather than automatically going down the pure Python path for stuff such as resolving names and such. This was never done to explicitly import using something defined in the globals namespace (although this does lead to supporting it).

And if you go all the way back to Python 3.0 you will notice that getting __import__ from the builtins namespace only has been in place: https://hg.python.org/cpython/file/3.0/Python/ceval.c#l1959 .

So I think the key point here is that the bytecode for IMPORT_NAME doesn't match the docs. So the question is do we want to add the feature to Python 3 to look for import in the globals or would we rather leave it as is and fix the docs?
msg253642 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-10-28 23:15
Fix the docs.
msg255891 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-12-04 22:52
New changeset 567baf74ebad by Brett Cannon in branch '3.5':
Issue #25500: Fix the language reference to not claim that import
https://hg.python.org/cpython/rev/567baf74ebad

New changeset 0259c2c555fb by Brett Cannon in branch 'default':
Merge for issue #25500
https://hg.python.org/cpython/rev/0259c2c555fb
msg255892 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-12-04 22:54
Thanks for the bug report, Sergei! I fixed the docs in the end.
History
Date User Action Args
2015-12-04 22:54:12brett.cannonsetstatus: open -> closed
resolution: fixed
messages: + msg255892

stage: resolved
2015-12-04 22:52:15python-devsetnosy: + python-dev
messages: + msg255891
2015-12-02 21:48:34brett.cannonsetassignee: docs@python -> brett.cannon
2015-10-28 23:15:46r.david.murraysetnosy: + r.david.murray
messages: + msg253642
2015-10-28 22:41:56brett.cannonsetmessages: + msg253638
title: _find_and_load_unlocked doesn't always use __import__ -> docs claim __import__ checked for in globals, but IMPORT_NAME bytecode does not
2015-10-28 21:37:23r.david.murraysetnosy: + brett.cannon, eric.snow
2015-10-28 21:34:44superbobrycreate