classification
Title: assertion failure in Python/ast.c in case of a bad unicodedata.normalize()
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Oren Milman, benjamin.peterson, brett.cannon, ncoghlan, serhiy.storchaka, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2017-09-26 12:54 by Oren Milman, last changed 2017-09-30 19:54 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3767 merged Oren Milman, 2017-09-26 14:40
PR 3836 merged python-dev, 2017-09-30 17:16
Messages (4)
msg303036 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-09-26 12:54
The following code causes an assertion failure:

import unicodedata
def bad_normalize(*args):
    return None

unicodedata.normalize = bad_normalize
import ast
ast.parse('\u03D5')


This is because init_normalization() (in Python/ast.c) assumes that
unicodedata.normalize() is valid, and stores it in the compiling struct.
Later, new_identifier() calls the stored function, assumes it returned a
string, and passes it to PyUnicode_InternInPlace(), which asserts it is a
string.
msg303043 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-26 14:36
Oh, and passing arguments to this function is tricky. If faked unicodedata.normalize save the args tuple it will see a mutated tuple containing a hanging reference.

This is a good opportunity of using the fast call protocol.
msg303416 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-30 17:16
New changeset 7dc46d8cf5854d9f4ce3271b29c21aea4872e8ad by Serhiy Storchaka (Oren Milman) in branch 'master':
bpo-31592: Fix an assertion failure in Python parser in case of a bad unicodedata.normalize(). (#3767)
https://github.com/python/cpython/commit/7dc46d8cf5854d9f4ce3271b29c21aea4872e8ad
msg303420 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-30 19:51
New changeset a4dfe1c9eaca6faf206f63a178233ffea208c906 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31592: Fix an assertion failure in Python parser in case of a bad unicodedata.normalize(). (GH-3767) (#3836)
https://github.com/python/cpython/commit/a4dfe1c9eaca6faf206f63a178233ffea208c906
History
Date User Action Args
2017-09-30 19:54:36serhiy.storchakasetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.6
2017-09-30 19:51:42serhiy.storchakasetmessages: + msg303420
2017-09-30 17:16:35python-devsetpull_requests: + pull_request3817
2017-09-30 17:16:29serhiy.storchakasetmessages: + msg303416
2017-09-26 14:40:15Oren Milmansetkeywords: + patch
stage: patch review
pull_requests: + pull_request3752
2017-09-26 14:36:07serhiy.storchakasetnosy: + yselivanov, benjamin.peterson, serhiy.storchaka, vstinner, brett.cannon, ncoghlan
messages: + msg303043
2017-09-26 12:54:08Oren Milmancreate