classification
Title: unexpected system error with pep420 style namespace packages
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.snow Nosy List: brett.cannon, eric.smith, eric.snow, python-dev, r.david.murray, ronaldoussoren
Priority: release blocker Keywords: patch

Created on 2015-05-14 11:58 by ronaldoussoren, last changed 2015-05-16 17:38 by eric.snow. This issue is now closed.

Files
File name Uploaded Description Edit
issue24192.diff eric.snow, 2015-05-16 00:30 review
Messages (8)
msg243181 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2015-05-14 11:58
The script below creates a basic PEP 420 style package with a single module in it ('package.sub') and tries to import that module

With 3.5 the script runs without problems and prints 42 (as expected). 

With a 3.5 (fresh checkout as of 2015-05-14) I get an SystemError:

$ python3.5 demo.py
Traceback (most recent call last):
  File "demo.py", line 10, in <module>
    import package.mod
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load
  File "<frozen importlib._bootstrap>", line 947, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 575, in module_from_spec
  File "<frozen importlib._bootstrap>", line 519, in _init_module_attrs
SystemError: Parent module '' not loaded, cannot perform relative import

#######
import os

os.mkdir('path1')
os.mkdir('path1/package')
with open('path1/package/mod.py', 'w') as fp:
    fp.write('print(42)\n')

import site
site.addsitedir('path1')
import package.mod
msg243186 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-05-14 12:24
I presume you meant that it works with 3.4?
msg243233 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-05-14 23:25
The problem is right where the traceback says.  Apparently there is a gap in the namespace package tests that I slipped through with my recent work to split out path-based import.  I'll work up a patch.
msg243243 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-05-15 04:26
Hmm, look like the test suite masks the issue due to the fact that importlib gets imported before running the applicable tests in test_namespace_pkgs.py.  This causes _frozen_importlib.__package__ to get set properly, thus masking the problem.

The problem is the use of relative imports in importlib._bootstrap.  The solution is to accomplish this in a different way.
msg243291 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-05-16 00:30
Here's a fix.  If I don't hear from anyone right away I'll push it in a few hours (or tomorrow morning).
msg243295 - (view) Author: Roundup Robot (python-dev) Date: 2015-05-16 03:57
New changeset 46b2c99121f5 by Eric Snow in branch 'default':
Issue #24192: Fix namespace package imports.
https://hg.python.org/cpython/rev/46b2c99121f5
msg243297 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2015-05-16 07:52
I can confirm that this fixes the problem I was having, both for the sample code I posted and for the real codebase I extracted this from.

Thanks.
msg243340 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-05-16 17:38
Great!  The buildbots are happy too. :)
History
Date User Action Args
2015-05-16 17:38:43eric.snowsetstatus: open -> closed

messages: + msg243340
2015-05-16 07:52:04ronaldoussorensetstatus: pending -> open

messages: + msg243297
2015-05-16 03:57:49eric.snowsetstatus: open -> pending
type: behavior
resolution: fixed
stage: patch review -> resolved
2015-05-16 03:57:11python-devsetnosy: + python-dev
messages: + msg243295
2015-05-16 00:30:54eric.snowsetassignee: eric.snow
2015-05-16 00:30:42eric.snowsetfiles: + issue24192.diff
keywords: + patch
messages: + msg243291

stage: patch review
2015-05-15 14:50:06brett.cannonsetpriority: normal -> release blocker
2015-05-15 04:26:55eric.snowsetmessages: + msg243243
2015-05-14 23:25:17eric.snowsetmessages: + msg243233
2015-05-14 13:41:57eric.smithsetnosy: + eric.smith
2015-05-14 12:24:48r.david.murraysetnosy: + eric.snow, r.david.murray, brett.cannon
messages: + msg243186
2015-05-14 11:58:40ronaldoussorencreate