classification
Title: Py_Decref on value crash the interpreter in Python/ceval.c:1104
Type: crash Stage: patch review
Components: Interpreter Core Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, Dennis Sweeney, eric.smith, sbz
Priority: normal Keywords: patch

Created on 2020-11-21 07:20 by sbz, last changed 2020-11-23 20:41 by BTaskaya.

Files
File name Uploaded Description Edit
crash.py sbz, 2020-11-21 07:20 crash poc
crash-report-txt.zip sbz, 2020-11-21 07:30
add-new-crasher-to-cover-multiple-versions.diff sbz, 2020-11-21 09:25
Pull Requests
URL Status Linked Edit
PR 23448 open sbz, 2020-11-21 17:00
Messages (7)
msg381527 - (view) Author: Sofian Brabez (sbz) * Date: 2020-11-21 07:20
This PoC is causing a local crash of python interpreters version 2.7,3.6,3.7,3.8 and 3.9.

By creating a code object of size 0 with a POP_TOP opcode, in Python/ceval.c the call to Py_DECREF(value) on a NULL pointer lead to a segmentation fault of the python interpreter.

It was tested on all python3.x versions against a fresh compilation of a git clone github.com/python/cpython.git on branches and master. You need to adapt the code() constructor because the parameters are different across versions but crash remains.

I'm just covering the version 3.7 in following text

$ git clone --depth 1 https://github.com/python/cpython.git
$ git checkout -b 3.7 origin/3.7
$ export CFLAGS+="-g -O0"
$ ./configure
$ make
$ ./python -V
Python 3.7.9+
$ ./python -c 'import sys; print(sys.version)'
3.7.9+ (heads/3.7-dirty:08ba61dade, Nov 21 2020, 04:57:20) 
[Clang 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611a
$ ./python crash.py

Running the python3.7 execution into gdb, helped me to locate the crash for python3.7 https://github.com/python/cpython/blob/3.7/Python/ceval.c#L1104

$ gdb --batch --silent ./python -ex 'r crash.py'
Program received signal SIGSEGV, Segmentation fault.
0x000000000033873a in _PyEval_EvalFrameDefault (f=0x800bdda00, throwflag=0) at Python/ceval.c:1104
1104                Py_DECREF(value);

Also I have executed the PoC on different platforms Linux, FreeBSD and MacOSX. The behaviour is the same and SIGSEGV the interpreter.

I have located the issue in the source code but I'm wondering what will be the best solution to fix it? Python developers should know better, I am open to your advices and suggestions.

I have noticed that one assertion handle this case (in master) https://github.com/python/cpython/blob/master/Python/ceval.c#L1430 but most of the interpreters are built without --with-assertions enabled, so the crash will still persist.

More details on this gist https://gist.github.com/sbz/267d35de5766c53835c5c4ef45b18705

I think the python interpreter shouldn't crash and handle properly this edge case.
msg381528 - (view) Author: Sofian Brabez (sbz) * Date: 2020-11-21 07:30
Linux, FreeBSD and MacOSX crash reports and backtraces joined in the zip.

Contributor Agreement 2020-09-23 signed.
msg381529 - (view) Author: Dennis Sweeney (Dennis Sweeney) * Date: 2020-11-21 08:32
From https://github.com/python/cpython/blob/master/Lib/test/crashers/bogus_code_obj.py :

"""
Broken bytecode objects can easily crash the interpreter.
This is not going to be fixed.  It is generally agreed that there is no
point in writing a bytecode verifier and putting it in CPython just for
this.  Moreover, a verifier is bound to accept only a subset of all safe
bytecodes, so it could lead to unnecessary breakage.
For security purposes, "restricted" interpreters are not going to let
the user build or load random bytecodes anyway.  Otherwise, this is a
"won't fix" case.
"""

import types

co = types.CodeType(0, 0, 0, 0, 0, b'\x04\x71\x00\x00',
                    (), (), (), '', '', 1, b'')
exec(co)
msg381536 - (view) Author: Sofian Brabez (sbz) * Date: 2020-11-21 09:25
Thanks Dennis for pointing me the crashers, I was not aware of them.

I have added a new crash test to cover multiple crashes in different interpreter versions. Current bogus does not crash where it could be possible to crash in older and newer interpreter versions.

Do you think it's worth to add it? I joined the patch against master and if yes I could submit a new PR.
msg381620 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-11-22 16:24
I'm sorry to interrupt but what is the exact reasoning behind adding a new, (I presume) redundant crasher? There are tons of different ways to crash the interpreter with malformed bytecode, how would adding only one of them bring any good?
msg381688 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-11-23 18:53
@BTaskaya: do you think this is too similar to bogus_code_obj.py? That's the only crasher I can see it being similar to.
msg381697 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-11-23 20:41
> do you think this is too similar to bogus_code_obj.py? That's the only crasher I can see it being similar to.

As far as I assume, yes, that is the generic VM crasher via custom code object execution. I feel its existence is good enough to answer to issue openings like this.
History
Date User Action Args
2020-11-23 20:41:21BTaskayasetmessages: + msg381697
2020-11-23 18:53:55eric.smithsetmessages: + msg381688
versions: + Python 3.10, - Python 3.6, Python 3.7
2020-11-22 16:24:47BTaskayasetnosy: + BTaskaya
messages: + msg381620
2020-11-21 21:03:12eric.smithsetnosy: + eric.smith
2020-11-21 17:00:42sbzsetstage: patch review
pull_requests: + pull_request22340
2020-11-21 09:26:00sbzsetfiles: + add-new-crasher-to-cover-multiple-versions.diff
keywords: + patch
messages: + msg381536
2020-11-21 08:32:44Dennis Sweeneysetnosy: + Dennis Sweeney
messages: + msg381529
2020-11-21 07:30:06sbzsetfiles: + crash-report-txt.zip

messages: + msg381528
2020-11-21 07:20:07sbzcreate