This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author erlendaasland
Recipients Mark.Shannon, erlendaasland, kj
Date 2021-05-21.11:35:25
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1621596925.93.0.331333541057.issue44032@roundup.psfhosted.org>
In-reply-to
Content
Using Pablo's (or Victor's) reproducer from bpo-44184, I'm getting a crash, apparently because _PyThreadState_PushLocals() is called after PyThreadState_Clear().

In Python/pystate.c interpreter_clear(), we're first calling PyThreadState_Clear() (line 295), and a few lines later (line 326) we're running _PyGC_CollectNoFail(). Clearing tstate _after_ the last GC collect resolves the issue (see attached patch), but it might introduce other problems; I'm not at all familiar with this part of the codebase.


FYI, git HEAD is at b11a951f16f0603d98de24fee5c023df83ea552c on main.


=================================================================
==22475==ERROR: AddressSanitizer: heap-use-after-free on address 0x629000000220 at pc 0x00010c985aa8 bp 0x7ffee3bab620 sp 0x7ffee3bab618
WRITE of size 8 at 0x629000000220 thread T0
    #0 0x10c985aa7 in _PyThreadState_PushLocals pystate.c:2030
    #1 0x10c7fc8b6 in _PyEval_Vector ceval.c:5164
    #2 0x10c397ebe in _PyFunction_Vectorcall call.c:342
    #3 0x10c5591b2 in _PyObject_VectorcallTstate abstract.h:114
    #4 0x10c558f6b in PyObject_CallOneArg abstract.h:184
    #5 0x10c5586c9 in call_unbound_noarg typeobject.c:1625
    #6 0x10c57d578 in slot_tp_finalize typeobject.c:7762
    #7 0x10ca2ea0b in finalize_garbage gcmodule.c:982
    #8 0x10ca2694d in gc_collect_main gcmodule.c:1287
    #9 0x10ca25f9c in _PyGC_CollectNoFail gcmodule.c:2123
    #10 0x10c97abed in interpreter_clear pystate.c:326
    #11 0x10c97ae81 in _PyInterpreterState_Clear pystate.c:358
    #12 0x10c9697d9 in finalize_interp_clear pylifecycle.c:1634
    #13 0x10c96925b in Py_FinalizeEx pylifecycle.c:1812
    #14 0x10ca1f143 in Py_RunMain main.c:668
    #15 0x10ca203e4 in pymain_main main.c:696
    #16 0x10ca20694 in Py_BytesMain main.c:720
    #17 0x10c055a91 in main python.c:15
    #18 0x7fff20582f3c in start+0x0 (libdyld.dylib:x86_64+0x15f3c)

0x629000000220 is located 32 bytes inside of 16384-byte region [0x629000000200,0x629000004200)
freed by thread T0 here:
    #0 0x10d518609 in wrap_free+0xa9 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x48609)
    #1 0x10c50a7fc in _PyObject_ArenaFree obmalloc.c:174
    #2 0x10c5098e7 in _PyObject_VirtualFree obmalloc.c:564
    #3 0x10c9808cf in PyThreadState_Clear pystate.c:904
    #4 0x10c97a178 in interpreter_clear pystate.c:295
    #5 0x10c97ae81 in _PyInterpreterState_Clear pystate.c:358
    #6 0x10c9697d9 in finalize_interp_clear pylifecycle.c:1634
    #7 0x10c96925b in Py_FinalizeEx pylifecycle.c:1812
    #8 0x10ca1f143 in Py_RunMain main.c:668
    #9 0x10ca203e4 in pymain_main main.c:696
    #10 0x10ca20694 in Py_BytesMain main.c:720
    #11 0x10c055a91 in main python.c:15
    #12 0x7fff20582f3c in start+0x0 (libdyld.dylib:x86_64+0x15f3c)

previously allocated by thread T0 here:
    #0 0x10d5184c0 in wrap_malloc+0xa0 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x484c0)
    #1 0x10c50a7d8 in _PyObject_ArenaMalloc obmalloc.c:168
    #2 0x10c5098af in _PyObject_VirtualAlloc obmalloc.c:558
    #3 0x10c985bb7 in allocate_chunk pystate.c:617
    #4 0x10c97d6ee in new_threadstate pystate.c:678
    #5 0x10c97cb49 in PyThreadState_New pystate.c:706
    #6 0x10c96e26b in pycore_create_interpreter pylifecycle.c:614
    #7 0x10c96ca7c in pyinit_config pylifecycle.c:834
    #8 0x10c967c3a in pyinit_core pylifecycle.c:1003
    #9 0x10c967429 in Py_InitializeFromConfig pylifecycle.c:1188
    #10 0x10ca2409f in pymain_init main.c:66
    #11 0x10ca201ee in pymain_main main.c:687
    #12 0x10ca20694 in Py_BytesMain main.c:720
    #13 0x10c055a91 in main python.c:15
    #14 0x7fff20582f3c in start+0x0 (libdyld.dylib:x86_64+0x15f3c)

SUMMARY: AddressSanitizer: heap-use-after-free pystate.c:2030 in _PyThreadState_PushLocals
Shadow bytes around the buggy address:
  0x1c51fffffff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c5200000000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c5200000010: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c5200000020: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c5200000030: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c5200000040: fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd
  0x1c5200000050: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c5200000060: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c5200000070: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c5200000080: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c5200000090: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==22475==ABORTING
History
Date User Action Args
2021-05-21 11:35:25erlendaaslandsetrecipients: + erlendaasland, Mark.Shannon, kj
2021-05-21 11:35:25erlendaaslandsetmessageid: <1621596925.93.0.331333541057.issue44032@roundup.psfhosted.org>
2021-05-21 11:35:25erlendaaslandlinkissue44032 messages
2021-05-21 11:35:25erlendaaslandcreate