Issue31166
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2017-08-10 02:04 by geeknik, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
31166.txt | geeknik, 2017-08-10 07:51 | valgrind -q ./python 31166.py |
Messages (10) | |||
---|---|---|---|
msg300034 - (view) | Author: geeknik (geeknik) | Date: 2017-08-10 02:04 | |
Python 3.7 git commit 3ca9f50 compiled with afl-clang-fast on Ubuntu 16 x64. The following script triggers undefined-behavior followed by a null pointer dereference and a segfault. import gc t0ing0=object() class A(object): def f():0 x=t0ing0 r=gc.get_referrers(t0ing0) if[0]:dct=r[0] a=A for i in range(1):a.f dct["f"]=lambda:0 (a.f) Objects/dictobject.c:547:12: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:547:12 in Objects/dictobject.c:1105:18: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1105:18 in Objects/dictobject.c:2739:15: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2739:15 in Objects/dictobject.c:789:27: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:789:27 in Objects/dictobject.c:1104:18: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1104:18 in Objects/dictobject.c:994:15: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:994:15 in Objects/dictobject.c:683:11: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:683:11 in Objects/dictobject.c:1024:9: runtime error: index 64 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1024:9 in Objects/dictobject.c:2882:31: runtime error: index 64 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2882:31 in Objects/dictobject.c:2346:15: runtime error: index 128 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2346:15 in Objects/dictobject.c:1449:11: runtime error: index 32 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1449:11 in Objects/dictobject.c:744:27: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:744:27 in Objects/dictobject.c:1631:22: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1631:22 in Objects/dictobject.c:554:31: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:554:31 in Objects/dictobject.c:1183:15: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:1183:15 in Objects/dictobject.c:835:27: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:835:27 in Objects/dictobject.c:2036:10: runtime error: index 128 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:2036:10 in Objects/dictobject.c:3504:38: runtime error: index 16 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:3504:38 in Objects/dictobject.c:3422:38: runtime error: index 64 out of bounds for type 'int8_t [8]' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:3422:38 in Objects/obmalloc.c:1258:36: runtime error: load of misaligned address 0xffffffffffffffff for type 'block *' (aka 'unsigned char *'), which requires 8 byte alignment 0xffffffffffffffff: note: pointer points here <memory cannot be printed> SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/obmalloc.c:1258:36 in ASAN:DEADLYSIGNAL ================================================================= ==768==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000506f9f bp 0x7fff0666cbc0 sp 0x7fff0666cb20 T0) ==768==The signal is caused by a READ memory access. ==768==Hint: address points to the zero page. #0 0x506f9e in _PyObject_Alloc /root/cpython/Objects/obmalloc.c:1258:36 #1 0x9a669b in PyUnicode_New /root/cpython/Objects/unicodeobject.c:1296:24 #2 0x9fea51 in _PyUnicodeWriter_PrepareInternal /root/cpython/Objects/unicodeobject.c:13561:26 #3 0x9b9db4 in PyUnicode_DecodeUTF8Stateful /root/cpython/Objects/unicodeobject.c:4995:9 #4 0x9c0e65 in _PyUnicode_FromId /root/cpython/Objects/unicodeobject.c:2115:22 #5 0x89561e in _PyObject_GetAttrId /root/cpython/Objects/object.c:850:23 #6 0x6b3f4a in _PyObject_CallMethodId /root/cpython/Objects/call.c:1086:16 #7 0x518569 in flush_std_files /root/cpython/Python/pylifecycle.c:889:15 #8 0x517942 in Py_FinalizeEx /root/cpython/Python/pylifecycle.c:959:9 #9 0x5a882b in Py_Main /root/cpython/Modules/main.c:921:9 #10 0x500382 in main /root/cpython/./Programs/python.c:102:11 #11 0x7f94db0bb3f0 in __libc_start_main /build/glibc-mXZSwJ/glibc-2.24/csu/../csu/libc-start.c:291 #12 0x433e49 in _start (/root/cpython/python+0x433e49) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV /root/cpython/Objects/obmalloc.c:1258:36 in _PyObject_Alloc ==768==ABORTING |
|||
msg300050 - (view) | Author: Benjamin Peterson (benjamin.peterson) * | Date: 2017-08-10 06:49 | |
Yeah, bad things happen when the underlying class dict, which isn't normally exposed to Python, is mutated. |
|||
msg300051 - (view) | Author: Inada Naoki (methane) * | Date: 2017-08-10 07:02 | |
I think it's false positive of ASAN. We have dynamically sized block. https://github.com/python/cpython/blob/3b0f620c1a2a21272a9e2aeca6ca1d1ac10f8162/Objects/dict-common.h#L49-L69 dictobject.c:547 calls memcpy to fill the block and head pointer is defined as `int8_t [8]`. ASAN doesn't know this is overallocated memory block. |
|||
msg300058 - (view) | Author: geeknik (geeknik) | Date: 2017-08-10 07:51 | |
So if I leave UBSan and ASan out of the equation and compile with gcc and run this script: Program received signal SIGSEGV, Segmentation fault. update_refs (containers=<optimized out>) at Modules/gcmodule.c:353 353 _PyGCHead_SET_REFS(gc, Py_REFCNT(FROM_GC(gc))); (gdb) bt #0 update_refs (containers=<optimized out>) at Modules/gcmodule.c:353 #1 collect (generation=generation@entry=2, n_collected=n_collected@entry=0x7fffffffe2f8, n_uncollectable=n_uncollectable@entry=0x7fffffffe300, nofail=nofail@entry=0) at Modules/gcmodule.c:962 #2 0x00005555555d5365 in collect_with_callback (generation=2) at Modules/gcmodule.c:1135 #3 PyGC_Collect () at Modules/gcmodule.c:1622 #4 _PyGC_CollectIfEnabled () at Modules/gcmodule.c:1635 #5 0x00005555555b8e28 in Py_FinalizeEx () at Python/pylifecycle.c:978 #6 0x00005555555b9225 in Py_FinalizeEx () at Python/pylifecycle.c:1119 #7 0x00005555555d2ed2 in Py_Main (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:921 #8 0x00005555555aa1cb in main (argc=2, argv=<optimized out>) at ./Programs/python.c:102 (gdb) list 348 update_refs(PyGC_Head *containers) 349 { 350 PyGC_Head *gc = containers->gc.gc_next; 351 for (; gc != containers; gc = gc->gc.gc_next) { 352 assert(_PyGCHead_REFS(gc) == GC_REACHABLE); 353 _PyGCHead_SET_REFS(gc, Py_REFCNT(FROM_GC(gc))); 354 /* Python's cyclic gc should never see an incoming refcount 355 * of 0: if something decref'ed to 0, it should have been 356 * deallocated immediately at that time. 357 * Possible cause (if the assert triggers): a tp_dealloc Valgrind shows a null deref as well after some invalid reads and conditional jumps. I've attached the log, it's a bit verbose. |
|||
msg300061 - (view) | Author: Inada Naoki (methane) * | Date: 2017-08-10 08:27 | |
As Benjamin commented, this is caused by mutating internal dict. PyType_Lookup() use "method cache", based on "tp_version_tag" in the type object. When you modify internal dict directly, namespace is changed without invalidating tp_version_tag. So cached pointer is used, and it's already deallocated. I don't know we should fix it or not. I don't have any idea fix this without any performance penalty. |
|||
msg300062 - (view) | Author: STINNER Victor (vstinner) * | Date: 2017-08-10 08:38 | |
> I don't know we should fix it or not. I don't have any idea fix this without any performance penalty. The PEP 509 (dict version) might help if we want to fix this bug. |
|||
msg300063 - (view) | Author: Inada Naoki (methane) * | Date: 2017-08-10 08:49 | |
But we should check dicts of all parents. It will has significant penalty, especially for classes having long mro (inheriting metaclass from typing module cause long mro). |
|||
msg300064 - (view) | Author: STINNER Victor (vstinner) * | Date: 2017-08-10 09:02 | |
"But we should check dicts of all parents. It will has significant penalty, especially for classes having long mro (inheriting metaclass from typing module cause long mro)." Oh right. That would defeat the whole purpose of the cache. Maybe we should not fix the bug. You are not supposed to access the hidden dictionary :-) |
|||
msg300065 - (view) | Author: STINNER Victor (vstinner) * | Date: 2017-08-10 09:03 | |
Another solution is to replace class dict with a special type which invalidates the type cache on dict[key]=value. |
|||
msg322484 - (view) | Author: Inada Naoki (methane) * | Date: 2018-07-27 10:28 | |
We ignores several crashes caused by exposing internal object through gc or weakref module. This case is not special enough to care. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:58:49 | admin | set | github: 75349 |
2018-07-27 10:28:55 | methane | set | status: open -> closed resolution: wont fix messages: + msg322484 stage: resolved |
2017-08-10 09:03:27 | vstinner | set | messages: + msg300065 |
2017-08-10 09:02:41 | vstinner | set | messages: + msg300064 |
2017-08-10 08:49:51 | methane | set | messages: + msg300063 |
2017-08-10 08:38:57 | vstinner | set | nosy:
+ vstinner messages: + msg300062 |
2017-08-10 08:27:17 | methane | set | messages: + msg300061 |
2017-08-10 07:51:30 | geeknik | set | files:
+ 31166.txt messages: + msg300058 |
2017-08-10 07:02:32 | methane | set | nosy:
+ methane messages: + msg300051 |
2017-08-10 06:49:53 | benjamin.peterson | set | nosy:
+ benjamin.peterson messages: + msg300050 |
2017-08-10 02:04:37 | geeknik | create |