Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bugs in PyImport_ImportModuleLevelObject #71606

Closed
serhiy-storchaka opened this issue Jun 30, 2016 · 10 comments
Closed

Bugs in PyImport_ImportModuleLevelObject #71606

serhiy-storchaka opened this issue Jun 30, 2016 · 10 comments
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@serhiy-storchaka
Copy link
Member

BPO 27419
Nosy @brettcannon, @ncoghlan, @ericsnowcurrently, @berkerpeksag, @serhiy-storchaka
Files
  • fix-import.patch
  • import_bug_1.py
  • import_bug_2.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-08-03.05:16:26.853>
created_at = <Date 2016-06-30.08:28:21.453>
labels = ['interpreter-core', 'type-crash']
title = 'Bugs in PyImport_ImportModuleLevelObject'
updated_at = <Date 2018-08-03.05:16:26.851>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2018-08-03.05:16:26.851>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2018-08-03.05:16:26.853>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-06-30.08:28:21.453>
creator = 'serhiy.storchaka'
dependencies = []
files = ['43585', '43586', '43621']
hgrepos = []
issue_num = 27419
keywords = ['patch']
message_count = 10.0
messages = ['269564', '269565', '269805', '269813', '270624', '270630', '270631', '270632', '322993', '323021']
nosy_count = 6.0
nosy_names = ['brett.cannon', 'ncoghlan', 'python-dev', 'eric.snow', 'berker.peksag', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue27419'
versions = ['Python 3.5', 'Python 3.6']
    @serhiy-storchaka
    Copy link
    Member Author

    1. If __package__ is None, and __spec__ is None or not set, and emitting ImportWarning raises an error, package is a borrowed reference to None, and it is decrefed. As result, the reference count of None is decremented.

    2. __import__ is looked in globals or builtins and passed to _find_and_load(), _handle_fromlist() or called with highest package name. This doesn't match Python implementation. Python implementation uses _gcd_import which implements the original __import__, and isn't affected by overriding __import__ in globals or builtins.

    @serhiy-storchaka serhiy-storchaka added type-bug An unexpected behavior, bug, or error interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Jun 30, 2016
    @serhiy-storchaka
    Copy link
    Member Author

    Here is a reproducer for the first bug.

    In Python 3.6:

    $ ./python import_bug_1.py
1738
738
Fatal Python error: deallocating None

    Current thread 0xb7552700 (most recent call first):
    Aborted (core dumped)

    In Python 3.5:

    $ ./python import_bug_1.py
1709
Traceback (most recent call last):
  File "import_bug_1.py", line 10, in <module>
    from .foo import bar
SystemError: Parent module '' not loaded, cannot perform relative import

    @serhiy-storchaka serhiy-storchaka added type-crash A hard crash of the interpreter, possibly with a core dump and removed type-bug An unexpected behavior, bug, or error labels Jun 30, 2016
    @serhiy-storchaka
    Copy link
    Member Author

    Here is a reproducer for the second bug.

    In Python 2.7 and Python 3.2 all works.

    In Python 3.3+:

    $ ./python import_bug_2.py 
Traceback (most recent call last):
  File "import_bug_2.py", line 6, in <module>
    import os.path
  File "import_bug_2.py", line 4, in __import__
    raise AssertionError
AssertionError

    @brettcannon
    Copy link
    Member

    The semantics for looking up __import__ is to only grab it from builtins and to ignore globals() and locals() (see issue bpo-25500), so any fix/change should follow those semantics.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jul 17, 2016

    New changeset c88ec1bb67d0 by Serhiy Storchaka in branch '3.5':
    Issue bpo-27419: Standard __import__() no longer look up "__import__" in globals
    https://hg.python.org/cpython/rev/c88ec1bb67d0

    New changeset d87f99e297d5 by Serhiy Storchaka in branch 'default':
    Issue bpo-27419: Standard __import__() no longer look up "__import__" in globals
    https://hg.python.org/cpython/rev/d87f99e297d5

    @berkerpeksag
    Copy link
    Member

    I'm getting a segfault after d87f99e297d5 (SubinterpreterTest.test_subinterps).

    $ ./python -m test test_capi
Run tests sequentially
0:00:00 [1/1] test_capi
Fatal Python error: Segmentation fault

    Current thread 0x00007fd5e401d700 (most recent call first):
    File "/home/berker/projects/cpython/default/Lib/test/support/init.py", line 2440 in run_in_subinterp
    File "/home/berker/projects/cpython/default/Lib/test/test_capi.py", line 337 in test_subinterps
    File "/home/berker/projects/cpython/default/Lib/unittest/case.py", line 600 in run
    File "/home/berker/projects/cpython/default/Lib/unittest/case.py", line 648 in __call__
    File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 122 in run
    File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 84 in __call__
    File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 122 in run
    File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 84 in __call__
    File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 122 in run
    File "/home/berker/projects/cpython/default/Lib/unittest/suite.py", line 84 in __call__
    File "/home/berker/projects/cpython/default/Lib/test/support/init.py", line 1704 in run
    File "/home/berker/projects/cpython/default/Lib/test/support/init.py", line 1805 in _run_suite
    File "/home/berker/projects/cpython/default/Lib/test/support/init.py", line 1839 in run_unittest
    File "/home/berker/projects/cpython/default/Lib/test/libregrtest/runtest.py", line 179 in test_runner
    File "/home/berker/projects/cpython/default/Lib/test/libregrtest/runtest.py", line 180 in runtest_inner
    File "/home/berker/projects/cpython/default/Lib/test/libregrtest/runtest.py", line 144 in runtest
    File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 334 in run_tests_sequential
    File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 406 in run_tests
    File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 466 in _main
    File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 446 in main
    File "/home/berker/projects/cpython/default/Lib/test/libregrtest/main.py", line 508 in main
    File "/home/berker/projects/cpython/default/Lib/test/main.py", line 2 in <module>
    File "/home/berker/projects/cpython/default/Lib/runpy.py", line 85 in _run_code
    File "/home/berker/projects/cpython/default/Lib/runpy.py", line 184 in _run_module_as_main
    Segmentation fault (core dumped)

    See also http://buildbot.python.org/all/builders/s390x%20RHEL%203.x/builds/1387/steps/test/logs/stdio for the same failure on buildbots.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jul 17, 2016

    New changeset 4c07faa33915 by Serhiy Storchaka in branch '3.5':
    Issue bpo-27419: Added temporary workaround for subinterpreters.
    https://hg.python.org/cpython/rev/4c07faa33915

    New changeset 5540234ca517 by Serhiy Storchaka in branch 'default':
    Issue bpo-27419: Added temporary workaround for subinterpreters.
    https://hg.python.org/cpython/rev/5540234ca517

    @serhiy-storchaka
    Copy link
    Member Author

    Added temporary workaround to make buildbots working. The bug is partially reappeared in __import__ in subinterpreter. I think this is better than reverting all patches.

    @berkerpeksag
    Copy link
    Member

    The workaround was removed in berkerpeksag@133138a Can this issue be closed now?

    @serhiy-storchaka
    Copy link
    Member Author

    I think it can. Thanks Berker for reminder.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @berkerpeksag @brettcannon @serhiy-storchaka