classification
Title: __import__ now raises with non-existing items in fromlist in 3.3
Type: behavior Stage: resolved
Components: Interpreter Core, Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: brett.cannon, eric.snow, georg.brandl, pitrou, python-dev, sfeltman
Priority: release blocker Keywords: patch

Created on 2012-08-16 22:42 by sfeltman, last changed 2012-08-18 03:38 by eric.snow. This issue is now closed.

Files
File name Uploaded Description Edit
issue15715.diff eric.snow, 2012-08-17 05:35 review
Messages (14)
msg168423 - (view) Author: Simon Feltman (sfeltman) Date: 2012-08-16 22:42
This came up while trying to build pygobject with Python 3.3. The problem is there are some erroneous imports in the fromlist with these bindings that can easily be fixed. But it is a behavioral change so I just wanted to raise awareness if it is not already known.

$ python3.2 -c "__import__('http', fromlist=['blah'])"
(works)

$ python3.3 -c "__import__('http', fromlist=['blah'])"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named 'http.blah'


Note this is also the case when using the C API: PyImport_ImportModuleEx
msg168424 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-08-16 23:14
While this is a regression, 3.3 definitely puts an increased emphasis on using importlib.import_module() instead of builtins.__import__().  Any reason why import_module() isn't used?

As to the regression, while the current behavior makes more sense, it is contrary to a common idiom when using __import__().  However, given the push away from using __import__(), perhaps it can just stay this way.  Oh backward compatibility...
msg168425 - (view) Author: Simon Feltman (sfeltman) Date: 2012-08-16 23:31
I think pygobject still supports Python 2.5 which it look like importlib is available.

I've submitted a patch to pygobject which will work regardless of if this regression is fixed or not: https://bugzilla.gnome.org/show_bug.cgi?id=682051
msg168426 - (view) Author: Simon Feltman (sfeltman) Date: 2012-08-16 23:32
Should have been "...Python 2.5 which it looks like importlib is NOT available."
msg168427 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-08-16 23:57
Unless it is difficult to fix, I think this regression should be addressed before 3.3 final. Georg?
msg168432 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-08-17 05:04
The failure output with -v:

...
# /home/esnow/projects/cpython/Lib/http/__pycache__/__init__.cpython-33.pyc matches /home/esnow/projects/cpython/Lib/http/__init__.py
# code object from /home/esnow/projects/cpython/Lib/http/__pycache__/__init__.cpython-33.pyc
import 'http' # <_frozen_importlib.SourceFileLoader object at 0x7ff82770d450>
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1578, in _handle_fromlist
  File "<frozen importlib._bootstrap>", line 310, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1530, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1494, in _find_and_load_unlocked
ImportError: No module named 'http.blah'
...
msg168433 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-08-17 05:35
Here's a simple patch the allows bogus names in the fromlist.  I'm going to verify that this matches the 3.2 semantics, which may not be so cut-and-dry.
msg168435 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-08-17 06:01
The following seems to indicate that an ImportError should be raised as expected.  I'm guessing that somewhere along the line the exception gets silently eaten.

------------------

(3.2) Python/import.c:ensure_fromlist() [1]

            submod = import_submodule(mod, subname, buf);
            Py_DECREF(item8);
            Py_XDECREF(submod);
            if (submod == NULL) {
                Py_DECREF(item);
                return 0;
            }

[1] http://hg.python.org/cpython/file/3.2/Python/import.c#l2875
msg168436 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-08-17 06:19
I agree that we should match 3.2 behavior here.
msg168447 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-08-17 16:08
And this is why we have said people should use the idiom of ``__import__('http.blah'); mod = sys.modules['http.blah']`` if importlib is not used (which is on PyPI and works as far back as Python 2.3), else you will deal with an AttributeError later instead of an ImportError upfront. <grumble>Darn backwards-compatibility</grumble>.

Eric's patch looks fine, so I will get it committed today.
msg168448 - (view) Author: Roundup Robot (python-dev) Date: 2012-08-17 17:21
New changeset 0d52f125dd32 by Brett Cannon in branch 'default':
Issue #15715: Ignore failed imports triggered by the use of fromlist.
http://hg.python.org/cpython/rev/0d52f125dd32
msg168450 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-08-17 17:36
When people want to import modules with "runtime" names, they regrettably turn to __import__() and likely will for a while.  What a source of headaches!

If it were less convenient to use __import__(), perhaps fewer people would use it.  Could we remove it from <module>.__builtins__, so that it would not be found during the lookup chain?  We could still expose it in the builtins module or move it to imp.
msg168453 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-08-17 17:54
We might be able to hide it in Python 3 since importlib.import_module() has been available since Python 3.1 (and 3.0 is dead anyway).
msg168494 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-08-18 03:38
I've taken the tanget over to issue15720.
History
Date User Action Args
2012-08-18 03:38:49eric.snowsetmessages: + msg168494
2012-08-17 17:54:31brett.cannonsetmessages: + msg168453
2012-08-17 17:36:03eric.snowsetmessages: + msg168450
2012-08-17 17:21:49brett.cannonsetstatus: open -> closed
resolution: fixed
stage: commit review -> resolved
2012-08-17 17:21:23python-devsetnosy: + python-dev
messages: + msg168448
2012-08-17 16:08:34brett.cannonsetassignee: brett.cannon
messages: + msg168447
stage: needs patch -> commit review
2012-08-17 06:19:24georg.brandlsetmessages: + msg168436
2012-08-17 06:01:42eric.snowsetmessages: + msg168435
2012-08-17 05:35:33eric.snowsetfiles: + issue15715.diff
keywords: + patch
messages: + msg168433
2012-08-17 05:04:22eric.snowsetmessages: + msg168432
2012-08-16 23:57:22pitrousetnosy: + georg.brandl
2012-08-16 23:57:15pitrousetpriority: normal -> release blocker

nosy: + pitrou
messages: + msg168427

components: + Interpreter Core, Library (Lib), - None
stage: needs patch
2012-08-16 23:32:35sfeltmansetmessages: + msg168426
2012-08-16 23:31:22sfeltmansetmessages: + msg168425
2012-08-16 23:14:42eric.snowsetnosy: + eric.snow, brett.cannon
messages: + msg168424
2012-08-16 22:50:31sfeltmansettype: behavior
2012-08-16 22:42:08sfeltmancreate