This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: import hook behavior documentation improvement
Type: behavior Stage:
Components: Documentation Versions: Python 2.7
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: docs@python Nosy List: brett.cannon, docs@python, iko, ncoghlan
Priority: normal Keywords:

Created on 2012-07-08 10:38 by iko, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
testimport.py iko, 2012-07-08 10:38 demonstration, see description
Messages (6)
msg164998 - (view) Author: Anders Hammarquist (iko) Date: 2012-07-08 10:38
When testing Eutaxia on PyPy (1.9) I discovered a discrepancy in the path_hooks import hook implementation. In CPython (2.7), if the find_module() method raises ImportError (as imp.find_module() does when it does not find a module in the given path), will cause the search to continue, whereas PyPy would propagate the ImportError.

PyPy has now been changed to behave like CPython.

The documentation is not entirely clear, but it does not explicitly document the import hook mechanism as eating an ImportError in find_module(). It should probably be made explicit, which ever way it should be. It is not obvious what is the correct behaviour, given the implicit relative imports, where the ImportError simply means that the import hook cannot find the module.

Quick testing on CPython 3.3 indicates that it behaves like PyPy did, but as it doesn't do implicit relative imports my test case didn't work as it was. For 3.3, without implicit relative imports, propagating the ImportError feels like the correct behaviour.

The attached demonstration needs a file /tmp/test/foo.py that does a top-level import, e.g. "import errno" to demonstrate the discrepancy.
msg165004 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2012-07-08 11:33
The PyPy and 3.3 behaviour are actually correct according to the spec, but it's *really* unclear in PEP 302.

sys.meta_path accepts finder objects. These are explicitly documented as returning "None" from find_module() to indicate "try the next one" and raising exceptions solely to report problems.

However, for reasons that are unknown to me, sys.path_hooks entries (which occupy most of the section on registering hooks) use a different protocol to indicate "try the next one": raising ImportError.

Since meta_path and path_hooks are described in the same section, and the meta_path description just says "add finder objects", it's understandable that implementors take the path_hooks protocol description as applying to finders in general :(

I would chalk the 2.x (and likely 3.x for x < 3) behaviour up to the only partial implementation of PEP 302 in CPython (until Brett's success in bootstrapping importlib for 3.3).
msg165015 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-07-08 13:40
Everything Nick said is right: PyPy did it properly according to the spec and CPython 2.7 got it wrong. Unfortunately fixing this now would break code and so it will simply have to stay a Python 2.7 quirk with Python 3.3 and later doing it correctly.

So documenting the screw-up would be good so that people know that the solution they use in Python 2.7 won't work in Python 3.3 and later.
msg165019 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2012-07-08 14:08
Perhaps the porting section in the 3.3 What's New?
msg165957 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-07-20 17:50
So I can't reproduce the problem under Python 2.7.3, Python 3.2.3, or a fresh checkout of Python 3.3. This is with both your script, Anders, and writing my own class that defined find_module() to just raise ImportError that I set in either sys.meta_path or in sys.path_importer_cache (both ways let the ImportError propagate).

This means I'm not sure exactly what problem you are encountering. Is this on an older version of Python 2.7? Do you have a unit test that can be run  which shows the test failing under Python 2.7 but passing under Python 3.3?
msg175769 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-11-17 16:44
Closing as "works for me" due to lack of reply from OP.
History
Date User Action Args
2022-04-11 14:57:32adminsetgithub: 59497
2012-11-17 16:44:58brett.cannonsetstatus: pending -> closed
resolution: works for me
messages: + msg175769
2012-07-20 17:50:48brett.cannonsetstatus: open -> pending

messages: + msg165957
2012-07-08 14:08:58ncoghlansetmessages: + msg165019
2012-07-08 13:40:58brett.cannonsetmessages: + msg165015
2012-07-08 11:33:21ncoghlansetmessages: + msg165004
2012-07-08 10:43:13pitrousetnosy: + brett.cannon, ncoghlan
2012-07-08 10:38:20ikocreate