classification
Title: `SystemError: returned NULL without setting an error` from importing _pickle on Windows
Type: behavior Stage: resolved
Components: Interpreter Core, Library (Lib), Windows Versions: Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: "SystemError: <class '_io.TextIOWrapper'> returned NULL without setting an error" from open function
View: 30626
Assigned To: Nosy List: brett.cannon, eric.snow, ncoghlan, paul.moore, ppperry, serhiy.storchaka, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords:

Created on 2017-07-08 02:29 by ppperry, last changed 2017-07-10 12:52 by vstinner. This issue is now closed.

Messages (5)
msg297924 - (view) Author: (ppperry) Date: 2017-07-08 02:29
The following rather long code, reduced from the same example as issue30626, produces a SystemError:

import builtins
from importlib.machinery import PathFinder
import importlib
import sys
import _imp
from functools import partial
def copy_module(module):
    new = type(sys)(module.__name__)
    new.__dict__.update(module.__dict__)
    return new
_absent = object()
def new_exec(code, globals=_absent, locals=_absent, *, new_builtins):
    if globals == _absent:
        frame = sys._getframe(2)
        globals = frame.f_globals
        locals = frame.f_locals
    elif locals == _absent:
        locals = globals
    globals.setdefault("__builtins__", new_builtins);
    return exec(code, globals, locals)
dct={}
dct["__builtins__"] = b = copy_module(builtins)
b.exec = partial(new_exec, new_builtins=b)
spec = PathFinder.find_spec("_bootstrap",importlib.__path__)
source_bootstrap = type(sys)("_bootstrap");
spec.loader.exec_module(source_bootstrap);
external_spec = PathFinder.find_spec("_bootstrap_external",importlib.__path__)
source_bootstrap_external = type(sys)("_bootstrap_external");
external_spec.loader.exec_module(source_bootstrap_external);
source_bootstrap.__name__ = "importlib._bootstrap";
source_bootstrap_external.__name__ = "importlib._bootstrap_external";
new_sys = copy_module(sys)
new_sys.path_importer_cache = {}
new_sys.path_hooks = []
new_sys.meta_path = []
new_sys.modules = {
                    "importlib._bootstrap":source_bootstrap,
                    "importlib._bootstrap_external":source_bootstrap_external,
                  }
for mod in new_sys.modules.values():
    mod.__builtins__ = b
b.__import__ = source_bootstrap.__import__
source_bootstrap._install(new_sys,_imp)
exec("import _pickle", dct)
msg297941 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-08 05:33
I get the following error on 3.7:

$ ./python issue30873.py 
Traceback (most recent call last):
  File "issue30873.py", line 44, in <module>
    exec("import _pickle", dct)
  File "<string>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/importlib/_bootstrap.py", line 1059, in __import__
    module = _gcd_import(name)
  File "/home/serhiy/py/cpython/Lib/importlib/_bootstrap.py", line 986, in _gcd_import
    return _find_and_load(name, _gcd_import)
  File "/home/serhiy/py/cpython/Lib/importlib/_bootstrap.py", line 963, in _find_and_load
    return _find_and_load_unlocked(name, import_)
  File "/home/serhiy/py/cpython/Lib/importlib/_bootstrap.py", line 948, in _find_and_load_unlocked
    raise ModuleNotFoundError(_ERR_MSG.format(name), name=name)
ModuleNotFoundError: No module named '_pickle'

On 3.6:

$ ./python issue30873.py 
Traceback (most recent call last):
  File "issue30873.py", line 44, in <module>
    exec("import _pickle", dct)
  File "<string>", line 1, in <module>
  File "/home/serhiy/py/cpython3.6/Lib/importlib/_bootstrap.py", line 1059, in __import__
    module = _gcd_import(name)
  File "/home/serhiy/py/cpython3.6/Lib/importlib/_bootstrap.py", line 986, in _gcd_import
    return _find_and_load(name, _gcd_import)
  File "/home/serhiy/py/cpython3.6/Lib/importlib/_bootstrap.py", line 963, in _find_and_load
    return _find_and_load_unlocked(name, import_)
  File "/home/serhiy/py/cpython3.6/Lib/importlib/_bootstrap.py", line 950, in _find_and_load_unlocked
    module = _load_unlocked(spec)
  File "/home/serhiy/py/cpython3.6/Lib/importlib/_bootstrap.py", line 648, in _load_unlocked
    module = module_from_spec(spec)
  File "/home/serhiy/py/cpython3.6/Lib/importlib/_bootstrap.py", line 560, in module_from_spec
    module = spec.loader.create_module(spec)
  File "<frozen importlib._bootstrap_external>", line 922, in create_module
  File "/home/serhiy/py/cpython3.6/Lib/importlib/_bootstrap.py", line 205, in _call_with_frames_removed
    return f(*args, **kwds)
KeyError: 'copyreg'
msg297953 - (view) Author: (ppperry) Date: 2017-07-08 13:27
Interesting. For you, `_pickle` seems to be an extension module, which is thus trying to call `imp.create_dynamic`, whereas for me it is a builtin module. Perhaps that explains why you get a KeyError and I get a SystemError (my traceback ends with):
  File "C:\Program Files\Python36\lib\importlib\_bootstrap.py", line 560, in module_from_spec
    module = spec.loader.create_module(spec)
  File "C:\Program Files\Python36\lib\importlib\_bootstrap.py", line 725, in create_module
    return _call_with_frames_removed(_imp.create_builtin, spec)
  File "C:\Program Files\Python36\lib\importlib\_bootstrap.py", line 205, in _call_with_frames_removed
    return f(*args, **kwds)
SystemError: <built-in function create_builtin> returned NULL without setting an error
(and is the same as your 3.6 traceback up to that point)
msg297954 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-08 13:51
Ah, this is Windows-only _imp.create_builtin().
msg298062 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-10 12:52
ppperry: First, please create a Python script and attach it to the issue, it would make it easier to reproduce the bug. Second, which Python version did you test? I tested 3.6 and master Git branches on Windows, and I'm unable to reproduce the bug.

I'm able to reproduce the bug on Windows using Python 3.6.1, but this version still contains issue30626 bug. I'm unable to reproduce the bug using 3.6.2rc2, so to be, this bug is just a duplicate of issue30626.

Please try: https://www.python.org/downloads/release/python-362rc2/
History
Date User Action Args
2017-07-10 12:52:47vstinnersetstatus: open -> closed

superseder: "SystemError: <class '_io.TextIOWrapper'> returned NULL without setting an error" from open function

nosy: + vstinner
messages: + msg298062
resolution: duplicate
stage: resolved
2017-07-08 13:58:09ppperrysettitle: `SystemError: <built-in function create_builtin> returned NULL without setting an error` from importing _pickle -> `SystemError: <built-in function create_builtin> returned NULL without setting an error` from importing _pickle on Windows
2017-07-08 13:51:41serhiy.storchakasetnosy: + tim.golden, paul.moore, zach.ware, steve.dower
messages: + msg297954
components: + Windows
2017-07-08 13:27:49ppperrysetmessages: + msg297953
2017-07-08 05:33:08serhiy.storchakasetmessages: + msg297941
2017-07-08 05:21:16serhiy.storchakasetnosy: + serhiy.storchaka
2017-07-08 02:31:09ppperrysettitle: `SystemError: <built-in function create_builtin> returned NULL without setting an error` from imp.create_builtin -> `SystemError: <built-in function create_builtin> returned NULL without setting an error` from importing _pickle
2017-07-08 02:29:24ppperrycreate