classification
Title: Set __loader__ on modules imported by the C level
Type: behavior Stage: test needed
Components: Interpreter Core Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: asvetlov, barry, brett.cannon, eric.snow, python-dev, theller
Priority: normal Keywords:

Created on 2013-02-01 13:39 by brett.cannon, last changed 2013-02-01 21:47 by theller. This issue is now closed.

Messages (11)
msg181078 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-01 13:39
E.g. _frozen_importlib, builtins, signal.
msg181083 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-01 16:23
Should probably check the state of sys.modules in importlib._bootstrap._setup() and see if the missing modules are there and then just retroactively set __loader__ to BuiltinImporter (as is already done in that function for sys and _imp): http://hg.python.org/cpython/file/905b4e3cf6d0/Lib/importlib/_bootstrap.py#l1709 .
msg181091 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-01 18:47
All cases but signal can be fixed by importlib._bootstrap._setup() by simply iterating over sys.modules and setting __loader__. Nice and simple. Thanks, abstraction!

The problem is signal who gives the finger to abstraction. That module gets imported directly by C code which cheats by calling PyInit_signal() directly and then calling _PyImport_FixupBuiltin(). I tried setting __loader__ in there but I'm actually triggering a segfault in PyErr_Format()! This is probably because I/O streams are not set up yet at this point in interpreter startup. Anyway, I'm trying to simply not have the code cheat and instead just import the signal module through the proper abstractions.
msg181095 - (view) Author: Roundup Robot (python-dev) Date: 2013-02-01 19:07
New changeset 05747d3bcd9c by Brett Cannon in branch '3.3':
Issue #17098: Make sure every module has __loader__ defined.
http://hg.python.org/cpython/rev/05747d3bcd9c

New changeset 1f1a1b3cc416 by Brett Cannon in branch 'default':
Issue #17098: all modules should have __loader__
http://hg.python.org/cpython/rev/1f1a1b3cc416
msg181096 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-01 19:08
OK, so my solution for signal worked. All fixed now. If any other modules pop up with __loader__ not set from now on it's because they cheated on import. =)
msg181103 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-01 19:40
Looks like I spoke too soon. I realized I had not written a test so I did it quickly and it turns out that while this is fixed for 3.3 it isn't for 3.4.
msg181105 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2013-02-01 20:47
I have only tried my code with 3.4; but still get problems with the modules 'builtins' and '_frozenimportlib'.
msg181110 - (view) Author: Roundup Robot (python-dev) Date: 2013-02-01 21:37
New changeset 4a4688b865ff by Brett Cannon in branch '3.3':
Add a test for fix of issue #17098
http://hg.python.org/cpython/rev/4a4688b865ff

New changeset 19ea454ccdf7 by Brett Cannon in branch '3.3':
Issue #17098: Be more stringent of setting __loader__ on early imported
http://hg.python.org/cpython/rev/19ea454ccdf7

New changeset 306f066e6a33 by Brett Cannon in branch 'default':
Merge w/ 3.3 more fixes thanks to issue #17098
http://hg.python.org/cpython/rev/306f066e6a33
msg181111 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-01 21:40
OK, 3.3 and 3.4 now have tests and verify everything is working fine.
msg181113 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2013-02-01 21:46
I confirm that the bug is fixed, my test-code works now (a modulefinder using importlib instead of imp to find modules).
msg181114 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2013-02-01 21:47
Thanks for the very fast fix Brett :-)
History
Date User Action Args
2013-02-01 21:47:29thellersetmessages: + msg181114
2013-02-01 21:46:56thellersetmessages: + msg181113
2013-02-01 21:40:56brett.cannonsetstatus: open -> closed

messages: + msg181111
2013-02-01 21:37:59python-devsetmessages: + msg181110
2013-02-01 20:47:45thellersetmessages: + msg181105
2013-02-01 19:42:51eric.snowsetnosy: + eric.snow
2013-02-01 19:40:54brett.cannonsetstatus: closed -> open

messages: + msg181103
2013-02-01 19:08:48brett.cannonsetstatus: open -> closed
assignee: brett.cannon
resolution: fixed
messages: + msg181096
2013-02-01 19:07:43python-devsetnosy: + python-dev
messages: + msg181095
2013-02-01 18:47:58brett.cannonsetmessages: + msg181091
2013-02-01 16:23:48brett.cannonsetmessages: + msg181083
2013-02-01 15:17:03barrysetnosy: + barry
2013-02-01 14:07:06asvetlovsetnosy: + asvetlov
2013-02-01 13:39:27brett.cannoncreate