This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: address sanitizer build fails
Type: compile error Stage: resolved
Components: Build Versions:
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: mdk, pmatos, vstinner
Priority: normal Keywords:

Created on 2017-08-14 09:21 by pmatos, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
files.zip pmatos, 2017-08-14 09:21 Zip with stdout and config.log
Pull Requests
URL Status Linked Edit
PR 13168 mdk, 2019-06-15 00:06
Messages (6)
msg300240 - (view) Author: Paulo Matos (pmatos) Date: 2017-08-14 09:21
Build with address sanitizer fails miserably.

Configuration in Fedora 26. I attach config.log and the output of 
$ ../configure --with-assertions --with-lto --with-pydebug --with-address-sanitizer --disable-ipv6
$ make -j18 profile-opt
msg300243 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2017-08-14 13:46
Are you sure you build is failing? It looks like it succeeded, if we only speak about the build.

You're seeing AddressSanitizer reports because, during the build, the built python is used. Like in `./python -E -S -m sysconfig --generate-posix-vars`.

So if you see AddressSanitizer output, it's because a `./python` has been successfully built and used.

The built cpython with AddressSanitizer however reports leaks, which is probably reproductible by running, typically:

  ./python -c ''
msg300246 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2017-08-14 14:14
Some (most?) of them looks to be unicode strings, and according to the very end of the main function:

#ifdef __INSURE__
    /* Insure++ is a memory analysis tool that aids in discovering
     * memory leaks and other memory problems.  On Python exit, the
     * interned string dictionaries are flagged as being in use at exit
     * (which it is).  Under normal circumstances, this is fine because
     * the memory will be automatically reclaimed by the system.  Under
     * memory debugging, it's a huge source of useless noise, so we
     * trade off slower shutdown for less distraction in the memory
     * reports.  -baw
     */
    _Py_ReleaseInternedUnicodeStrings();
#endif /* __INSURE__ */

So, to ensure everything is as expected, I compiled with the __INSURE__ flag, and got:

releasing 1894 interned strings
total size of all interned strings: 17945/0 mortal/immortal
ASAN:DEADLYSIGNAL
=================================================================
==23814==ERROR: AddressSanitizer: SEGV on unknown address 0x0000000000a0 (pc 0x5575b1c16904 bp 0x7ffe1c4f1f60 sp 0x7ffe1c4f1f40 T0)
    #0 0x5575b1c16903 in list_dealloc (/home/mdk/Downloads/cpython/python+0x524903)
    #1 0x5575b1bf287d in _Py_Dealloc (/home/mdk/Downloads/cpython/python+0x50087d)
    #2 0x5575b1c3daca in _Py_ReleaseInternedUnicodeStrings (/home/mdk/Downloads/cpython/python+0x54baca)
    #3 0x5575b206c0d3 in Py_Main (/home/mdk/Downloads/cpython/python+0x97a0d3)
    #4 0x5575b206d1d2 in main (/home/mdk/Downloads/cpython/python+0x97b1d2)
    #5 0x7f5ed0b622b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
    #6 0x5575b1b1f8d9 in _start (/home/mdk/Downloads/cpython/python+0x42d8d9)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/home/mdk/Downloads/cpython/python+0x524903) in list_dealloc
==23814==ABORTING

Which in this case does not looks normal.
msg300253 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2017-08-14 15:28
Segfault occur in Include/object.h line 1054:

#define Py_TRASHCAN_SAFE_BEGIN(op) \
    do { \
        PyThreadState *_tstate = PyThreadState_GET(); \
        if (_tstate->trash_delete_nesting < PyTrash_UNWIND_LEVEL) { \
            ⧺_tstate->trash_delete_nesting;
            /* The body of the deallocator is here. */

the _tstate is null at this moment.

It's NULL because in the main, right before the _Py_ReleaseInternedUnicodeStrings there's a call to Py_FinalizeEx which calls PyThreadState_Swap(NULL), see pylifecycle.c:1097:

    /* Delete current thread. After this, many C API calls become crashy. */
    PyThreadState_Swap(∅);

But there's probably still references to strings before the Py_FinalizeEx so it does not make sense to garbage collect before it.

Maybe make Py_TRASHCAN_SAFE_BEGIN more robust by disabling the counter when the state is NULL?
msg341800 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2019-05-07 19:03
Tried to reproduce it again, on master, but this does no longer segfault nor report leakages (as long as I use __INSURE__ using make -j18 profile-opt CFLAGS='-D__INSURE__') nor fail at build time.

There's still a few memory warnings while running tests (subinterpreters and things).

So I'm closing this (no build issue), thanks for reporting.
msg364590 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-19 02:26
FYI there is bpo-1635741 which tries to cleanup more things at Python exit.
History
Date User Action Args
2022-04-11 14:58:50adminsetgithub: 75383
2020-03-19 02:26:28vstinnersetnosy: + vstinner
messages: + msg364590
2019-06-15 00:06:08mdksetpull_requests: + pull_request13952
2019-05-07 19:03:43mdksetstatus: open -> closed
resolution: works for me
messages: + msg341800

stage: resolved
2017-08-14 15:28:47mdksetmessages: + msg300253
2017-08-14 14:14:34mdksetmessages: + msg300246
2017-08-14 13:46:21mdksetnosy: + mdk
messages: + msg300243
2017-08-14 09:21:33pmatoscreate