classification
Title: relative import without parent succeeds with builtins.__import__
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: Ben Lewis2, brett.cannon, eric.smith, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2019-06-26 09:11 by Ben Lewis2, last changed 2019-08-02 22:11 by brett.cannon.

Pull Requests
URL Status Linked Edit
PR 14465 closed Ben Lewis2, 2019-06-29 15:49
PR 14956 open Ben Lewis2, 2019-07-26 08:29
Messages (8)
msg346593 - (view) Author: Ben Lewis (Ben Lewis2) * Date: 2019-06-26 09:11
>>> from curses import ascii
>>> from . import ascii

The second line should raise an ImportError but instead succeeds (tested cpython 3.6.7, 3.7.0 and 3.7.3, and from interactive interpreter and scripts). Specifically, builtins.__import__ does not reproduce the behaviour of importlib._bootstrap.__import__; maybe ceval.c:import_from is neglecting to check that there are parent packages when attempting a relative import?

More details here: https://stackoverflow.com/a/56768129/5104777
msg346654 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-06-26 18:05
I can't reproduce:

>>> from importlib import abc
>>> from . import util
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'util' from '__main__' (unknown location)
>>> import importlib.util
>>>

Did you happen to do an `import *` prior to this in your REPL? If so that might explain it as you could have pulled in __package__ and such and that's used to resolve relative imports.
msg346710 - (view) Author: Ben Lewis (Ben Lewis2) * Date: 2019-06-27 04:45
>>> foo = 'oops'
>>> from . import foo as fubar   # should raise ImportError
>>> fubar
'oops'

After further investigation, the problem is that builtins.__import__ (the c port version) does not replicate the behaviour of importlib.__import__ (the python reference version):

>>> import builtins, importlib
>>> __package__ is None
True
>>> importlib.__import__('', globals(), locals(), ('foo',), 1)
ImportError
>>> builtins.__import__('', globals(), locals(), ('foo',), 1)
<module '__main__' (built-in)>

A further discrepancy is that for deeper relative imports, builtins.__import__ raises a ValueError instead of ImportError (contrary to expectation/spec):

>>> from ...... import foo
ValueError

A simple work around uses the python implementation to restore expected behaviour:

>>> builtins.__import__ = importlib.__import__
>>> from ...... import foo
ImportError
>>> from curses import ascii
>>> from . import ascii
ImportError

PS: Brett Cannon, to replicate please copy and paste lines in correct order :-)
msg346781 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-06-27 22:01
Please open a separate issue for the relative import issue.
msg346856 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-06-28 19:22
I opened bpo-37444 for the relative import beyond top-level package issue.
msg346857 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-06-28 19:25
The code doing the sanity check for importlib can be found at https://github.com/python/cpython/blob/f9f8e3ce709ceb15c8db8c8dde940daf1febf13d/Lib/importlib/_bootstrap.py#L943-L948 .
msg346861 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-06-28 19:34
If you run with `-Xdev`/warnings turned on you get an idea of what's happening:

>>> builtins.__import__('', globals(), locals(), ('foo',), 1)
<stdin>:1: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
<module '__main__' (built-in)>

The check is being done in resolve_name() in import.c (https://github.com/python/cpython/blob/f9f8e3ce709ceb15c8db8c8dde940daf1febf13d/Python/import.c#L1543). My guess is there's an off-by-one error in the sanity check logic for attempting a relative import beyond the top-level package.
msg346866 - (view) Author: Ben Lewis (Ben Lewis2) * Date: 2019-06-28 21:12
I'll look into this further and open a PR initially for the regression tests.
History
Date User Action Args
2019-08-02 22:11:28brett.cannonsetassignee: brett.cannon
2019-07-26 08:29:52Ben Lewis2setpull_requests: + pull_request14724
2019-06-29 15:49:49Ben Lewis2setkeywords: + patch
stage: test needed -> patch review
pull_requests: + pull_request14282
2019-06-28 21:12:20Ben Lewis2setmessages: + msg346866
2019-06-28 19:34:59brett.cannonsetmessages: + msg346861
2019-06-28 19:33:51serhiy.storchakasetnosy: + serhiy.storchaka
2019-06-28 19:28:23brett.cannonsettitle: relative import without parent -> relative import without parent succeeds with builtins.__import__
2019-06-28 19:25:37brett.cannonsetmessages: + msg346857
2019-06-28 19:22:37brett.cannonsetmessages: + msg346856
2019-06-28 19:19:00brett.cannonsetversions: + Python 3.8, Python 3.9, - Python 3.7
2019-06-27 22:01:16brett.cannonsetmessages: + msg346781
stage: resolved -> test needed
2019-06-27 04:45:43Ben Lewis2setstatus: closed -> open
resolution: rejected ->
messages: + msg346710

title: relative import_from without parent -> relative import without parent
2019-06-26 18:05:28brett.cannonsetstatus: open -> closed
resolution: rejected
messages: + msg346654

stage: resolved
2019-06-26 09:31:21eric.smithsetnosy: + brett.cannon, eric.smith
2019-06-26 09:11:26Ben Lewis2create