classification
Title: "SystemError: returned NULL without setting an error" from open function
Type: crash Stage: resolved
Components: Interpreter Core, IO, Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: benjamin.peterson, brett.cannon, eric.snow, ncoghlan, ppperry, serhiy.storchaka, stutzbach
Priority: normal Keywords:

Created on 2017-06-10 18:53 by ppperry, last changed 2017-06-15 18:21 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 2103 merged serhiy.storchaka, 2017-06-11 08:13
PR 2221 merged serhiy.storchaka, 2017-06-15 17:56
PR 2222 merged serhiy.storchaka, 2017-06-15 17:57
Messages (7)
msg295654 - (view) Author: (ppperry) Date: 2017-06-10 18:57
This code (reduced from an example I had of trying to debug code including the module-level code of already imported modules), raises a SystemError:
import builtins
from importlib.machinery import PathFinder
from importlib.util import find_spec
import importlib
import sys
import _imp
dct={}
def copy_module(module):
    new = type(sys)(module.__name__)
    new.__dict__.update(module.__dict__)
    return new

dct["__builtins__"] = b = copy_module(builtins)

spec = PathFinder.find_spec("_bootstrap",importlib.__path__)
source_bootstrap = type(sys)("_bootstrap");
spec.loader.exec_module(source_bootstrap);
source_bootstrap.__name__ = "importlib._bootstrap";    
new_sys = copy_module(sys)

new_sys.path_hooks = []
new_sys.meta_path = []
new_sys.modules = {
    "importlib._bootstrap":source_bootstrap,
    "importlib._bootstrap_external":importlib._bootstrap_external,
}
b.__import__ = source_bootstrap.__import__
source_bootstrap._install(new_sys,_imp)
dct["__file__"]=__file__
exec("open(__file__)",dct)

The actual file passed to the open function doesn't matter, as long as it would work
msg295680 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-06-11 01:48
I couldn't reproduce this on 3.7, but I can confirm that 3.6 gives a SystemError with the above code:

$ ./python _issue30626.py
Traceback (most recent call last):
  File "_issue30626.py", line 30, in <module>
    exec("open(__file__)",dct)
  File "<string>", line 1, in <module>
SystemError: <class '_io.TextIOWrapper'> returned NULL without setting an error

It's plausible this was fixed as part of the interpreter startup refactoring, so I doubt we're going to able to easily isolate the specific change that fixed it :(
msg295681 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-06-11 01:49
That said: given that we know it *is* fixed somewhere in 3.7, it would likely be useful to check the assumption that the startup refactoring fixed it by going to the last commit before that landed and seeing if the error still occurs.
msg295693 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-11 08:16
In rare circumstances PyImport_Import() could return NULL without raising an error.

It *isn't* fixed in 3.7.
msg296118 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-15 17:54
New changeset 145541cfa05394c38cfd64c0be2c5fb382860995 by Serhiy Storchaka in branch 'master':
bpo-30626: Fix error handling in PyImport_Import(). (#2103)
https://github.com/python/cpython/commit/145541cfa05394c38cfd64c0be2c5fb382860995
msg296119 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-15 18:15
New changeset fab05de21480fc54a5101cf813195cb32724b5ad by Serhiy Storchaka in branch '3.6':
[3.6] bpo-30626: Fix error handling in PyImport_Import(). (GH-2103) (#2221)
https://github.com/python/cpython/commit/fab05de21480fc54a5101cf813195cb32724b5ad
msg296120 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-15 18:15
New changeset 263dcc39daa74066c2b2fcb007a4bd4f7ec65073 by Serhiy Storchaka in branch '3.5':
[3.5] bpo-30626: Fix error handling in PyImport_Import(). (GH-2103) (#2222)
https://github.com/python/cpython/commit/263dcc39daa74066c2b2fcb007a4bd4f7ec65073
History
Date User Action Args
2017-07-10 12:52:47vstinnerlinkissue30873 superseder
2017-06-15 18:21:52serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-06-15 18:15:29serhiy.storchakasetmessages: + msg296120
2017-06-15 18:15:13serhiy.storchakasetmessages: + msg296119
2017-06-15 17:57:00serhiy.storchakasetpull_requests: + pull_request2267
2017-06-15 17:56:07serhiy.storchakasetpull_requests: + pull_request2266
2017-06-15 17:54:40serhiy.storchakasetmessages: + msg296118
2017-06-11 08:16:09serhiy.storchakasetstage: needs patch -> patch review
messages: + msg295693
versions: + Python 3.7
2017-06-11 08:13:09serhiy.storchakasetpull_requests: + pull_request2157
2017-06-11 05:36:25serhiy.storchakasetassignee: serhiy.storchaka

type: behavior -> crash
nosy: + serhiy.storchaka
versions: + Python 3.5
2017-06-11 01:49:52ncoghlansetmessages: + msg295681
2017-06-11 01:48:12ncoghlansetstage: needs patch
messages: + msg295680
versions: + Python 3.6
2017-06-10 19:00:50ppperrysettype: behavior
2017-06-10 18:58:41ppperrysettitle: "SystemError: SystemError: <class '_io.TextIOWrapper'> returned NULL without setting an error" from open function -> "SystemError: <class '_io.TextIOWrapper'> returned NULL without setting an error" from open function
2017-06-10 18:58:28ppperrysetnosy: + brett.cannon, ncoghlan, benjamin.peterson, stutzbach, eric.snow
2017-06-10 18:57:30ppperrysetmessages: + msg295654
components: + Interpreter Core, Library (Lib), IO
title: "SystemError: SystemError: <class '_io.TextIOWrapper'> returned NULL without setting an error -> "SystemError: SystemError: <class '_io.TextIOWrapper'> returned NULL without setting an error" from open function
2017-06-10 18:53:58ppperrycreate