classification
Title: Bugs in PyImport_ImportModuleLevelObject
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: berker.peksag, brett.cannon, eric.snow, ncoghlan, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-06-30 08:28 by serhiy.storchaka, last changed 2018-08-03 05:16 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
fix-__import__.patch serhiy.storchaka, 2016-06-30 08:28 review
import_bug_1.py serhiy.storchaka, 2016-06-30 08:43
import_bug_2.py serhiy.storchaka, 2016-07-04 19:50
Messages (10)
msg269564 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-30 08:28
1. If __package__ is None, and __spec__ is None or not set, and emitting ImportWarning raises an error, package is a borrowed reference to None, and it is decrefed. As result, the reference count of None is decremented.

2. __import__ is looked in globals or builtins and passed to _find_and_load(), _handle_fromlist() or called with highest package name. This doesn't match Python implementation. Python implementation uses _gcd_import which implements the original __import__, and isn't affected by overriding __import__ in globals or builtins.
msg269565 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-30 08:43
Here is a reproducer for the first bug.

In Python 3.6:

$ ./python import_bug_1.py
1738
738
Fatal Python error: deallocating None

Current thread 0xb7552700 (most recent call first):
Aborted (core dumped)

In Python 3.5:

$ ./python import_bug_1.py
1709
Traceback (most recent call last):
  File "import_bug_1.py", line 10, in <module>
    from .foo import bar
SystemError: Parent module '' not loaded, cannot perform relative import
msg269805 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-04 19:50
Here is a reproducer for the second bug.

In Python 2.7 and Python 3.2 all works.

In Python 3.3+:

$ ./python import_bug_2.py 
Traceback (most recent call last):
  File "import_bug_2.py", line 6, in <module>
    import os.path
  File "import_bug_2.py", line 4, in __import__
    raise AssertionError
AssertionError
msg269813 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-07-04 22:24
The semantics for looking up __import__ is to only grab it from builtins and to ignore globals() and locals() (see issue #25500), so any fix/change should follow those semantics.
msg270624 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-07-17 09:52
New changeset c88ec1bb67d0 by Serhiy Storchaka in branch '3.5':
Issue #27419: Standard __import__() no longer look up "__import__" in globals
https://hg.python.org/cpython/rev/c88ec1bb67d0

New changeset d87f99e297d5 by Serhiy Storchaka in branch 'default':
Issue #27419: Standard __import__() no longer look up "__import__" in globals
https://hg.python.org/cpython/rev/d87f99e297d5
msg270630 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-07-17 10:43
I'm getting a segfault after d87f99e297d5 (SubinterpreterTest.test_subinterps).

$ ./python -m test test_capi
Run tests sequentially
0:00:00 [1/1] test_capi
Fatal Python error: Segmentation fault

Current thread 0x00007fd5e401d700 (most recent call first):
  File "/home/berker/projects/cpython/default/Lib/test/support/__init__.py", line 2440 in run_in_subinterp
  File "/home/berker/projects/cpython/default/Lib/test/test_capi.py", line 337 in test_subinterps
  File "/home/berker/projects/cpython/default/Lib/unittest/case.py", line 600 in run
  File "/home/berker/projects/cpython/default/Lib/unittest/case.py", line 648 in __call__
  File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 122 in run
  File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 84 in __call__
  File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 122 in run
  File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 84 in __call__
  File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 122 in run
  File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 84 in __call__
  File "/home/berker/projects/cpython/default/Lib/test/support/__init__.py", line 1704 in run
  File "/home/berker/projects/cpython/default/Lib/test/support/__init__.py", line 1805 in _run_suite
  File "/home/berker/projects/cpython/default/Lib/test/support/__init__.py", line 1839 in run_unittest
  File "/home/berker/projects/cpython/default/Lib/test/libregrtest/runtest.py", line 179 in test_runner
  File "/home/berker/projects/cpython/default/Lib/test/libregrtest/runtest.py", line 180 in runtest_inner
  File "/home/berker/projects/cpython/default/Lib/test/libregrtest/runtest.py", line 144 in runtest
  File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 334 in run_tests_sequential
  File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 406 in run_tests
  File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 466 in _main
  File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 446 in main
  File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 508 in main
  File "/home/berker/projects/cpython/default/Lib/test/__main__.py", line 2 in <module>
  File "/home/berker/projects/cpython/default/Lib/runpy.py", line 85 in _run_code
  File "/home/berker/projects/cpython/default/Lib/runpy.py", line 184 in _run_module_as_main
Segmentation fault (core dumped)

See also http://buildbot.python.org/all/builders/s390x%20RHEL%203.x/builds/1387/steps/test/logs/stdio for the same failure on buildbots.
msg270631 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-07-17 11:16
New changeset 4c07faa33915 by Serhiy Storchaka in branch '3.5':
Issue #27419: Added temporary workaround for subinterpreters.
https://hg.python.org/cpython/rev/4c07faa33915

New changeset 5540234ca517 by Serhiy Storchaka in branch 'default':
Issue #27419: Added temporary workaround for subinterpreters.
https://hg.python.org/cpython/rev/5540234ca517
msg270632 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-17 11:20
Added temporary workaround to make buildbots working. The bug is partially reappeared in __import__ in subinterpreter. I think this is better than reverting all patches.
msg322993 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2018-08-02 20:29
The workaround was removed in https://github.com/berkerpeksag/cpython/commit/133138a284be1985ebd9ec9014f1306b9a425d98 Can this issue be closed now?
msg323021 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-08-03 05:16
I think it can. Thanks Berker for reminder.
History
Date User Action Args
2018-08-03 05:16:26serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg323021

stage: patch review -> resolved
2018-08-02 20:29:46berker.peksagsetmessages: + msg322993
2016-07-17 11:20:46serhiy.storchakasetmessages: + msg270632
2016-07-17 11:16:34python-devsetmessages: + msg270631
2016-07-17 10:43:29berker.peksagsetnosy: + berker.peksag
messages: + msg270630
2016-07-17 09:52:31python-devsetnosy: + python-dev
messages: + msg270624
2016-07-04 22:24:19brett.cannonsetmessages: + msg269813
2016-07-04 19:50:22serhiy.storchakasetfiles: + import_bug_2.py

messages: + msg269805
2016-06-30 08:43:24serhiy.storchakasetfiles: + import_bug_1.py
type: behavior -> crash
messages: + msg269565
2016-06-30 08:28:21serhiy.storchakacreate