Title: importlib.import_module() bypasses builtins.__import__
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, terry.reedy
Priority: normal Keywords:

Created on 2013-08-25 15:16 by brett.cannon, last changed 2020-03-06 20:28 by brett.cannon. This issue is now closed.

Messages (9)
msg196144 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-08-25 15:16
Not sure if anyone really cares about this (I don't, but I figured I should at least let it be known), but I realized that importlib.import_module() bypasses builtins.__import__ by calling directly into the innards of importlib (specifically _gcd_import() which skips all the extra setup that __import__ entails and which is unnecessary for programmatic imports).

The docs for importlib.import_module() clearly state it uses importlib.__import__, but I'm not sure if anyone would be surprised if they replaced builtins.__import__ and found that importlib.import_module() was doing an end-run around their custom import system, especially since we are promoting importlib.import_module() over calling builtins.__import__ directly.
msg196574 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-30 21:00
The discrepancy between doc and code should be eliminated. In this case, it seems the doc should be changed (in 3.3 as well). Would 'not' in the right place suffice?
msg196587 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-08-30 22:13
Not sure what doc discrepancy you are talking about, Terry. The docs for importlib are totally accurate which means no one has been accidentally mislead by them. This bug is about whether importlib.import_module() bypassing builtins.__import__ is a big enough deal to change or not.
msg196594 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-30 23:04
Perhaps I am confused because I did not notice the switch between builtins.__import__ and importlib.__import__ -- and I do not know the relation between the two, if indeed they are two and not one. What I read and understood is
* importlib.import_module doc say it calls ???.__import__;
* importlib.import_module code does not call ???.__import__;
it instead calls _gcd_import (for good reasons).

If the claim and reality disagree, because ??? is effectively the same in both statements, then either code or doc could be changed.

If importlib.__import__ and builtins.__import__ are different objects, and the doc is clear about that, then no one should expect that 'calls importlib.__import__' means 'calls builtins.__import__'.
msg196637 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-08-31 13:14
builtins.__import__ and importlib.__import__ are different objects.

This isn't about whether this is whole situation is a bug per-se, but whether users expect that overriding builtins.__import__ will affect all import-related code in the stdlib and if that's reasonable.
msg196650 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-31 17:11
Adding something like '(not builtins.__import__)' or '(different from builtins.__import__') after 'importlib.__import__' should reduce the possibility of confusing the two and expecting something that will not happen.

You might check the doc for (builtins.)__import__ to make sure it does not promise anything that is no longer true.

What happens if people monkeypatch importlib.__import__?
msg196671 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-08-31 20:46
help(__import__) doesn't mention anything about overriding the function. I already touched up the stdlib docs for builtins.__import__ to strongly advise you don't override the function.

As for overriding importlib.__import__, it won't do anything; the code for importlib.import_module() cuts through the cruft in __import__ that is unneeded and skips straight to the relevant code.
msg196675 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-31 21:35
Looking at the actual doc:
"importlib.__import__(name, globals=None, locals=None, fromlist=(), level=0) An implementation of the built-in __import__() function."

I think this is pretty clear that builtin.__import__ has no effect unless directly called.

"importlib.import_module(name, package=None)...The import_module() function acts as a simplifying wrapper around importlib.__import__()."

The word 'wrapper' implies that import_module calls __import__, but you say it currently does not. So if you want to keep the code as is, I would change 'acts as a simplifying wrapper around' to 'is a simplified version of', which makes no promises.

Do you really want people to directly call importlib.__import__? If not, maybe the nearly empty entry should be deleted and the entry for import_module changed to make no reference to it.
msg363543 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-03-06 20:28
I'm calling it and saying people are not expecting these semantics.
Date User Action Args
2020-03-06 20:28:16brett.cannonsetstatus: open -> closed
resolution: not a bug
messages: + msg363543

stage: test needed -> resolved
2013-08-31 21:35:54terry.reedysetmessages: + msg196675
2013-08-31 20:46:08brett.cannonsetmessages: + msg196671
2013-08-31 17:11:55terry.reedysetmessages: + msg196650
2013-08-31 13:14:55brett.cannonsetmessages: + msg196637
2013-08-30 23:04:50terry.reedysetmessages: + msg196594
2013-08-30 22:13:10brett.cannonsetmessages: + msg196587
2013-08-30 21:00:47terry.reedysetnosy: + terry.reedy
messages: + msg196574
2013-08-25 15:16:41brett.cannoncreate