classification
Title: Wrong ImportError message with importlib
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: Arfrever, amaury.forgeotdarc, asvetlov, brett.cannon, chris.jerdonek, python-dev, r.david.murray, yselivanov
Priority: high Keywords: 3.2regression

Created on 2012-06-20 00:17 by amaury.forgeotdarc, last changed 2012-10-24 11:56 by asvetlov. This issue is now closed.

Messages (12)
msg163234 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-06-20 00:17
Up to Python3.2, a nested ImportError was correctly displayed:

./python -c "import distutils.msvc9compiler"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/amauryfa/python/cpython3.2/Lib/distutils/msvc9compiler.py", line 27, in <module>
    import winreg
ImportError: No module named winreg

But with 3.3, the traceback is lost:

~/python/cpython3.x$ ./python -c "from distutils import msvc9compiler"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: cannot import name msvc9compiler

Even though the failure is still on the "import winreg" line. Only ImportError seems affected, other exceptions are correctly displayed.
msg163240 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-06-20 01:03
To clarify Amaury's example:

rdmurray@hey:~/python/p32>./python -c "from distutils import msvc9compiler"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/rdmurray/python/p32/Lib/distutils/msvc9compiler.py", line 27, in <module>
    import winreg
ImportError: No module named winreg

rdmurray@hey:~/python/p33>./python -c "from distutils import msvc9compiler"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: cannot import name msvc9compiler

So there is definitely some lost information in 3.3 compared to 3.2 in the 'import from' case.
msg165186 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-07-10 13:07
That "cannot import name" message seems to only come from Python/ceval.c:import_from() which raises that exception when an AttributeError is raised by an 'import from' statement (I think). This happens when an 'import from' asks for a name on the package/module that doesn't exist.

Looking at importlib, I found the code that captures the ImportErrorof a submodule, which allows the import to continue and then fail as an AttributeError at the bytecode level. I'm running the test suite now with that try/except removed to see if this was just a mis-interpretation on my part of how to handle this situation. As of right now the only failures I have continue my belief that import * is evil.
msg165188 - (view) Author: Roundup Robot (python-dev) Date: 2012-07-10 14:05
New changeset dc18a2a66d16 by Brett Cannon in branch 'default':
Issue #15111: When a module was imported using a 'from import'
http://hg.python.org/cpython/rev/dc18a2a66d16
msg172079 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2012-10-05 13:12
I don't know why, but it seems that the bug reappeared in 3.3.

Examples:

1. --------------- (python 3.4 from repo)

yury@sxair ~/dev/py/python (master) $ ./python.exe 
Python 3.4.0a0 (default, Oct  5 2012, 15:08:35) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.65))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils import msvc9compiler
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name msvc9compiler

2. --------------- (python 3.3.0 from macports)

yury@sxair ~/dev/py/python (master) $ python3.3
Python 3.3.0 (default, Oct  5 2012, 13:41:24) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.65))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils import msvc9compiler
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name msvc9compiler


3. ----------- (python 3.2 from macports)

yury@sxair ~/dev/py/python (master) $ python3.2
Python 3.2.2 (default, Sep 27 2012, 13:31:01) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.65))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils import msvc9compiler
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/distutils/msvc9compiler.py", line 27, in <module>
    import winreg
ImportError: No module named winreg

Platform: macos x 10.8

Please reopen the issue.
msg172093 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-10-05 15:50
> I don't know why, but it seems that the bug reappeared in 3.3.

Part of it could be that the original fix added no tests.
msg172095 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-10-05 16:41
http://hg.python.org/cpython/rev/dc18a2a66d16 did add a test, just not the right one.
msg172096 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-10-05 16:42
Actually, I take it back, it removed a test without adding a new one. Obviously that's my bad.
msg172243 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2012-10-06 20:28
Brett, can we patch 3.3 too? (in 3.3.1 version?)
msg172479 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-10-09 13:12
This can get fixed in 3.3.1, which is why I left Python 3.3 as an affected version.

Hopefully I can get to a fix this week. I need to write a test showing that a module that doesn't exist as specified in a fromlist is silently ignored, but if a module in a fromlist does exist *but* triggers an ImportError itself while being imported that exception propagates. Then I need to come up with a fix.
msg172572 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-10-10 13:28
Here are two possible tests.

1) __import__('distutils', fromlist=['_i_do_not_exist']) should return distutils

2) Use a fake loader which executes some code which does nothing more than tries to import a non-existent module. The trick with this test is that the submodule must be found but a module that the submodule needs cannot be found by import itself (and simply not faked with an ImportError thanks to the _not_found hack and needing that attribute to propagate up to the submodule search).

And I think the failure stems from the lack of check against exc.name equaling the name of the module being imported (e.g. exc.name == distutils.msvc9compiler) since the winreg import is a failed module search as well.
msg172606 - (view) Author: Roundup Robot (python-dev) Date: 2012-10-10 23:18
New changeset 09b5158d5284 by Brett Cannon in branch '3.3':
Closes issue #15111: Calling __import__ with a module specified in
http://hg.python.org/cpython/rev/09b5158d5284

New changeset 31db8e3272f1 by Brett Cannon in branch 'default':
Merge fix for issue #15111.
http://hg.python.org/cpython/rev/31db8e3272f1
History
Date User Action Args
2012-10-24 11:56:04asvetlovsetnosy: + asvetlov
2012-10-10 23:19:14brett.cannonsetstatus: open -> closed
resolution: fixed
2012-10-10 23:18:49python-devsetmessages: + msg172606
2012-10-10 13:28:04brett.cannonsetmessages: + msg172572
2012-10-09 13:12:07brett.cannonsetmessages: + msg172479
2012-10-08 01:36:04Arfreversetnosy: + Arfrever
2012-10-06 20:28:15yselivanovsetmessages: + msg172243
2012-10-05 16:42:35brett.cannonsetmessages: + msg172096
2012-10-05 16:41:45brett.cannonsetmessages: + msg172095
2012-10-05 15:50:39chris.jerdoneksetmessages: + msg172093
2012-10-05 15:07:27brett.cannonsetkeywords: + 3.2regression
status: closed -> open
resolution: fixed -> (no value)
versions: + Python 3.4
2012-10-05 13:12:17yselivanovsetnosy: + yselivanov
messages: + msg172079
2012-08-22 00:15:24chris.jerdonekunlinkissue15316 superseder
2012-07-10 14:05:35brett.cannonsetstatus: open -> closed
assignee: brett.cannon
resolution: fixed
stage: needs patch -> resolved
2012-07-10 14:05:09python-devsetnosy: + python-dev
messages: + msg165188
2012-07-10 13:07:59brett.cannonsetmessages: + msg165186
2012-07-10 07:57:41chris.jerdoneksetnosy: + chris.jerdonek
2012-07-10 07:54:48amaury.forgeotdarclinkissue15316 superseder
2012-06-20 01:03:49r.david.murraysetmessages: + msg163240
2012-06-20 01:01:04r.david.murraysetmessages: - msg163238
2012-06-20 01:00:17r.david.murraysetpriority: normal -> high
versions: + Python 3.3
nosy: + r.david.murray, brett.cannon

messages: + msg163238

stage: needs patch
2012-06-20 00:17:25amaury.forgeotdarccreate