classification
Title: reload problem with submodule
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, Olivier.Grisel, brett.cannon, eric.snow, ncoghlan, python-dev, ronaldoussoren
Priority: critical Keywords: 3.3regression, patch

Created on 2013-12-01 14:11 by ronaldoussoren, last changed 2013-12-10 03:14 by eric.snow. This issue is now closed.

Files
File name Uploaded Description Edit
issue19851-fix-reload.diff eric.snow, 2013-12-08 05:46 review
Messages (7)
msg204920 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-12-01 14:11
To reproduce:

* create a package with the following structure:

    pkg/
      __init__.py
      _sub.py

* __init__.py contains:

   from pkg._sub import *

* the contents of _sub.py is not important

* in a python shell do:

>>> import pkg._sub as s
>>> import imp
>>> imp.reload(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/imp.py", line 297, in reload
    return importlib.reload(module)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/importlib/__init__.py", line 123, in reload
    raise ImportError(_bootstrap._ERR_MSG.format(name), name=name)
ImportError: No module named 'pkg._sub'
>>> 

In earlier python versions this reloaded the pkg._sub module.  Importing _sub from __init__ doesn't seem to be important.
msg205510 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-12-08 03:27
This is actually a problem with importlib.reload() (which imp.reload() simply wraps).  The attached patch provides a test that reproduces the error.  I'll work on a fix ASAP.

Interestingly, the kind of failure depends on frozen vs. source importlib:

======================================================================
ERROR: test_reload_submodule (test.test_importlib.test_api.Frozen_ReloadTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_importlib/test_api.py", line 430, in test_reload_submodule
    self.init.reload(ham)
  File "Lib/importlib/__init__.py", line 161, in reload
    methods.exec(module)
  File "<frozen importlib._bootstrap>", line 1134, in exec
AttributeError: 'NoneType' object has no attribute 'name'

======================================================================
ERROR: test_reload_submodule (test.test_importlib.test_api.Source_ReloadTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_importlib/test_api.py", line 430, in test_reload_submodule
    self.init.reload(ham)
  File "Lib/importlib/__init__.py", line 158, in reload
    raise ImportError(msg.format(parent_name), name=parent_name)
ImportError: parent 'spam' not in sys.modules

----------------------------------------------------------------------
msg205511 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-12-08 03:51
Actually, they're both getting the same error:

AttributeError: 'NoneType' object has no attribute 'name'

I forgot to clear the submodule from sys.modules first.
msg205517 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-12-08 05:46
The problem was that importlib.reload() was not passing the parent's __path__ to importlib._bootstrap._find_spec().  This was a consequence of us restoring the pre-3.3 reload semantics.

Patch attached.  (Note to self: add Misc/NEWS entry)
msg205519 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-12-08 06:21
Patch looks reasonable to me.
msg205705 - (view) Author: Olivier Grisel (Olivier.Grisel) * Date: 2013-12-09 15:34
I tested the patch on the current HEAD and it fixes a regression introduced between 3.3 and 3.4b1 that prevented to build scipy from source with "pip install scipy".
msg205767 - (view) Author: Roundup Robot (python-dev) Date: 2013-12-10 03:05
New changeset 1d67eb1df5a9 by Eric Snow in branch 'default':
Issue 19851: Fix a regression in reloading submodules.
http://hg.python.org/cpython/rev/1d67eb1df5a9
History
Date User Action Args
2013-12-10 03:14:04eric.snowsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2013-12-10 03:05:31python-devsetnosy: + python-dev
messages: + msg205767
2013-12-09 15:34:57Olivier.Griselsetnosy: + Olivier.Grisel
messages: + msg205705
2013-12-08 06:21:41ncoghlansetmessages: + msg205519
2013-12-08 05:46:40eric.snowsetfiles: - issue19851-test.diff
2013-12-08 05:46:12eric.snowsetfiles: + issue19851-fix-reload.diff

messages: + msg205517
stage: needs patch -> patch review
2013-12-08 03:51:26eric.snowsetmessages: + msg205511
2013-12-08 03:27:44eric.snowsetfiles: + issue19851-test.diff
priority: normal -> critical
title: imp.reload problem with submodule -> reload problem with submodule
messages: + msg205510

keywords: + patch
stage: test needed -> needs patch
2013-12-01 14:56:41Arfreversetnosy: + Arfrever
2013-12-01 14:11:34ronaldoussorencreate