importlib does not properly remove frames when invoking external import hooks #67961

shiz · 2015-03-25T03:28:58Z

BPO	23773
Nosy	@brettcannon, @ncoghlan, @ericsnowcurrently, @raulcd
Files	cpython-3ac58de829ef-fix-external-module-loader-call-traceback.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2015-03-25.03:28:58.344>
labels = ['type-bug', 'library']
title = 'importlib does not properly remove frames when invoking external import hooks'
updated_at = <Date 2016-09-08.17:28:36.728>
user = 'https://bugs.python.org/shiz'

bugs.python.org fields:

activity = <Date 2016-09-08.17:28:36.728>
actor = 'brett.cannon'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2015-03-25.03:28:58.344>
creator = 'shiz'
dependencies = []
files = ['38681']
hgrepos = []
issue_num = 23773
keywords = ['patch']
message_count = 3.0
messages = ['239219', '240764', '240781']
nosy_count = 5.0
nosy_names = ['brett.cannon', 'ncoghlan', 'eric.snow', 'shiz', 'raulcd']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue23773'
versions = ['Python 3.4', 'Python 3.5']

shiz · 2015-03-25T03:28:57Z

When adding a custom module loader to sys.meta_path, importlib does not properly remove its frames before invoking it. This results in a weird traceback with importlib._bootstrap frames in if an error occurs during load_module(), like such:

Traceback (most recent call last):
  File "/Users/mark/Development/Projects/rave/rave/bootstrap/__init__.py", line 102, in bootstrap_game
    __import__(MODULE_PACKAGE + '.' + mod.replace('.py', ''))
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/__init__.py", line 1, in <module>
    from . import common, core3
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/__init__.py", line 11, in <module>
    from .texture import Texture, Image
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/texture.py", line 8, in <module>
    raise ValueError

Attached is a patch against the current hg head that makes sure module loaders are invoked through _call_with_frames_removed, so that the output becomes a lot more readable:

Traceback (most recent call last):
  File "/Users/mark/Development/Projects/rave/rave/bootstrap/__init__.py", line 102, in bootstrap_game
    __import__(MODULE_PACKAGE + '.' + mod.replace('.py', ''))
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/__init__.py", line 1, in <module>
    from . import common, core3
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/__init__.py", line 11, in <module>
    from .texture import Texture, Image
  File "/Users/mark/Development/Projects/rave/rave/modularity.py", line 23, in exec_module
    super().exec_module(module)
  File "/.modules/opengl/core3/texture.py", line 8, in <module>
    raise ValueError
ValueError

Ideally it would remove the calls /within/ the module loader itself too, but I do not see an easy way to do this without changing the entire _call_with_frames_removed semantics. This fixes most of the issues, though.

brettcannon · 2015-04-13T21:33:27Z

Did you happen to run the test suite with this change, Shiz? There's actually a test to make sure that the part of the traceback you're avoiding isn't lost in case there's a bug in importlib itself.

Can you show that your change won't actually affect finding errors in importlib itself?

shiz · 2015-04-13T22:14:32Z

Ah, that is a very good point. I'll run the suite as soon as I find the time and will report back.

shiz mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Mar 25, 2015

brettcannon self-assigned this Mar 25, 2015

brettcannon removed their assignment Sep 8, 2016

ezio-melotti transferred this issue from another repository Apr 10, 2022

hauntsaninja added the topic-importlib label May 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

importlib does not properly remove frames when invoking external import hooks #67961

importlib does not properly remove frames when invoking external import hooks #67961

shiz mannequin commented Mar 25, 2015

shiz mannequin commented Mar 25, 2015

brettcannon commented Apr 13, 2015

shiz mannequin commented Apr 13, 2015

importlib does not properly remove frames when invoking external import hooks #67961

importlib does not properly remove frames when invoking external import hooks #67961

Comments

shiz mannequin commented Mar 25, 2015

shiz mannequin commented Mar 25, 2015

brettcannon commented Apr 13, 2015

shiz mannequin commented Apr 13, 2015