Trying to run make after './configure --with-address-sanitizer --with-pydebug' fails with leak of locale string

just for me, btharper, which system do you use? because I have a fedora 29 and when I try to compile with these flags, getaddrinfo is not found.

ok, found

I have to disable ipv6 and install libasan

./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
	echo "generate-posix-vars failed" ; \
	rm -f ./pybuilddir.txt ; \
	exit 1 ; \
fi

=================================================================
==5408==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 34 byte(s) in 1 object(s) allocated from:
    #0 0x7fb085e03c08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
    #1 0x4d99a2 in _PyMem_RawMalloc Objects/obmalloc.c:99
    #2 0x4d9b4a in _PyMem_DebugRawAlloc Objects/obmalloc.c:2020
    #3 0x4d9ba5 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2049
    #4 0x4dc354 in PyMem_RawMalloc Objects/obmalloc.c:527
    #5 0x4dc510 in _PyMem_RawStrdup Objects/obmalloc.c:613
    #6 0x652f5f in get_ctype_locale Python/preconfig.c:483
    #7 0x6560a6 in _PyPreConfig_Read Python/preconfig.c:506
    #8 0x616cde in _PyCoreConfig_ReadPreConfig Python/coreconfig.c:1342
    #9 0x620564 in _PyCoreConfig_Read Python/coreconfig.c:1378
    #10 0x65ef06 in pyinit_coreconfig Python/pylifecycle.c:740
    #11 0x65fa05 in _Py_InitializeCore Python/pylifecycle.c:786
    #12 0x42abba in pymain_init Modules/main.c:370
    #13 0x42addc in pymain_main Modules/main.c:889
    #14 0x42b2bb in _Py_UnixMain Modules/main.c:940
    #15 0x4258ee in main Programs/python.c:16
    #16 0x7fb085983412 in __libc_start_main ../csu/libc-start.c:308

Direct leak of 34 byte(s) in 1 object(s) allocated from:
    #0 0x7fb085e03c08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
    #1 0x4d99a2 in _PyMem_RawMalloc Objects/obmalloc.c:99
    #2 0x4d9b4a in _PyMem_DebugRawAlloc Objects/obmalloc.c:2020
    #3 0x4d9ba5 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2049
    #4 0x4dc354 in PyMem_RawMalloc Objects/obmalloc.c:527
    #5 0x4dc510 in _PyMem_RawStrdup Objects/obmalloc.c:613
    #6 0x652f5f in get_ctype_locale Python/preconfig.c:483
    #7 0x6560a6 in _PyPreConfig_Read Python/preconfig.c:506
    #8 0x6585f8 in pyinit_preconfig Python/pylifecycle.c:723
    #9 0x65f8b0 in _Py_InitializeCore Python/pylifecycle.c:781
    #10 0x42abba in pymain_init Modules/main.c:370
    #11 0x42addc in pymain_main Modules/main.c:889
    #12 0x42b2bb in _Py_UnixMain Modules/main.c:940
    #13 0x4258ee in main Programs/python.c:16
    #14 0x7fb085983412 in __libc_start_main ../csu/libc-start.c:308

SUMMARY: AddressSanitizer: 68 byte(s) leaked in 2 allocation(s).
generate-posix-vars failed
make: *** [Makefile:586: pybuilddir.txt] Error 1

your PR seems to be fine.

but I continue to get two refleaks with valgrind but don't worry, @vstinner is working on these refleaks.

==5440== 64 bytes in 1 blocks are possibly lost in loss record 1 of 2
==5440==    at 0x483880B: malloc (vg_replace_malloc.c:309)
==5440==    by 0x46BFAB: _PyMem_RawMalloc (obmalloc.c:99)
==5440==    by 0x46BEA3: _PyMem_DebugRawAlloc (obmalloc.c:2020)
==5440==    by 0x46BF33: _PyMem_DebugRawMalloc (obmalloc.c:2049)
==5440==    by 0x46D304: PyMem_RawMalloc (obmalloc.c:527)
==5440==    by 0x52C1EF: PyThread_allocate_lock (thread_pthread.h:330)
==5440==    by 0x5154F0: _PyRuntimeState_Init_impl (pystate.c:52)
==5440==    by 0x515A76: _PyRuntimeState_Init (pystate.c:77)
==5440==    by 0x514114: _PyRuntime_Initialize (pylifecycle.c:88)
==5440==    by 0x4236FE: pymain_init (main.c:339)
==5440==    by 0x423980: pymain_main (main.c:889)
==5440==    by 0x423A6A: _Py_UnixMain (main.c:940)
==5440== 
==5440== 64 bytes in 1 blocks are possibly lost in loss record 2 of 2
==5440==    at 0x483880B: malloc (vg_replace_malloc.c:309)
==5440==    by 0x46BFAB: _PyMem_RawMalloc (obmalloc.c:99)
==5440==    by 0x46BEA3: _PyMem_DebugRawAlloc (obmalloc.c:2020)
==5440==    by 0x46BF33: _PyMem_DebugRawMalloc (obmalloc.c:2049)
==5440==    by 0x46D304: PyMem_RawMalloc (obmalloc.c:527)
==5440==    by 0x52C1EF: PyThread_allocate_lock (thread_pthread.h:330)
==5440==    by 0x515506: _PyRuntimeState_Init_impl (pystate.c:58)
==5440==    by 0x515A76: _PyRuntimeState_Init (pystate.c:77)
==5440==    by 0x514114: _PyRuntime_Initialize (pylifecycle.c:88)
==5440==    by 0x4236FE: pymain_init (main.c:339)
==5440==    by 0x423980: pymain_main (main.c:889)
==5440==    by 0x423A6A: _Py_UnixMain (main.c:940)

New changeset e130a07eb20c4b655d182d5d10d778c7584efe55 by Victor Stinner (btharper) in branch 'master':
bpo-36356: Fix memory leak in _PyPreConfig_Read() (GH-12425)
https://github.com/python/cpython/commit/e130a07eb20c4b655d182d5d10d778c7584efe55

New changeset a712679a2bffffefaacdc05f788d6ea50f72a561 by Victor Stinner in branch 'master':
bpo-36333, bpo-36356: Fix _PyEval_FiniThreads() (GH-12432)
https://github.com/python/cpython/commit/a712679a2bffffefaacdc05f788d6ea50f72a561

New changeset fecc4f2b474f16062514e95a67e66080fd626e14 by Victor Stinner in branch 'master':
bpo-36356: Release Unicode interned strings on Valgrind (#12431)
https://github.com/python/cpython/commit/fecc4f2b474f16062514e95a67e66080fd626e14

New changeset f5f336a819a3d881bb217bf8f9b5cacba03a4e45 by Victor Stinner in branch 'master':
bpo-36356: pymain_free() calls _PyRuntime_Finalize() (GH-12435)
https://github.com/python/cpython/commit/f5f336a819a3d881bb217bf8f9b5cacba03a4e45

New changeset 935250d6f3ac7ba91e1ea8e6ca63aaf7f605e291 by Victor Stinner in branch '3.7':
bpo-36356: pymain_free() calls _PyRuntime_Finalize() (GH-12436)
https://github.com/python/cpython/commit/935250d6f3ac7ba91e1ea8e6ca63aaf7f605e291

At least, "./python -V" no longer leaks at commit dcf617152e1d4c4a5e7965733928858a9c0936ca.

$ ./configure --with-valgrind
$ make
$ valgrind ./python -V
==9553== Memcheck, a memory error detector
==9553== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==9553== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==9553== Command: ./python -V
==9553== 
Python 3.8.0a2+
==9553== 
==9553== HEAP SUMMARY:
==9553==     in use at exit: 0 bytes in 0 blocks
==9553==   total heap usage: 33 allocs, 33 frees, 5,933 bytes allocated
==9553== 
==9553== All heap blocks were freed -- no leaks are possible
==9553== 
==9553== For counts of detected and suppressed errors, rerun with: -v
==9553== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

I'm on Ubuntu 18.10/amd64 compiling with Ubuntu's GCC 8.2.0, I know there's some libraries that are missing dependencies (including bz2 and sqlite) so I may have missed the ipv6 dependencies as well.

My eventual goal was to be able to build a pgo optimized build of cpython with the address sanitizer turned on. Currently on my build of f5f336a819 I see 12 failing tests and a heap use after free as part of the ctypes test suite. 

The ctypes use after free has already been submitted as bpo-36253 since it was the only thing asan caught during tests with the leak sanitizer turned off.

New changeset 4a1468e593c4b67d8c78b78070fff9e18ec5d790 by Victor Stinner in branch 'master':
bpo-36356: Fix _PyCoreConfig_Read() (GH-12454)
https://github.com/python/cpython/commit/4a1468e593c4b67d8c78b78070fff9e18ec5d790

For what it's worth, with current HEAD of master (commit 62be33870e2f8517314bf9c7275548e799296f7e which includes previously merged PRs from this issue), current macOS clang with address sanitizer and pydebug enabled gets an assertion failure in parsetok.c.  Current HEAD of 3.7 does not.

$ ./configure --with-address-sanitizer --prefix=/tmp/d --with-pydebu
$ make -j3
[...]
./python -E -S -m sysconfig --generate-posix-vars ;\
	if test $? -ne 0 ; then \
		echo "generate-posix-vars failed" ; \
		rm -f ./pybuilddir.txt ; \
		exit 1 ; \
	fi
 CC='gcc' LDSHARED='gcc -bundle -undefined dynamic_lookup -fsanitize=address    ' OPT='-g -O0 -Wall' 	_TCLTK_INCLUDES='' _TCLTK_LIBS='' 	./python -E ./setup.py  build
Assertion failed: ((intptr_t)(int)(a - line_start) == (a - line_start)), function parsetok, file Parser/parsetok.c, line 308.
/bin/sh: line 1: 95059 Abort trap: 6           CC='gcc' LDSHARED='gcc -bundle -undefined dynamic_lookup -fsanitize=address    ' OPT='-g -O0 -Wall' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py $quiet build
make: *** [sharedmods] Error 134
$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1000.11.45.5)
Target: x86_64-apple-darwin18.2.0
Thread model: posix
# same result with explicit CC=clang

I'm not sure about the change from 3.7 to master, but that assertion may be more likely to happen while instrumented with ASAN due to the extra space reserved between heap objects. As far as I can tell it's just expecting that the offset of two pointers will fit within an int instead of a intptr_t (4 bytes versus 8 bytes on my system).

For me running the test_pydoc from the test suite fails reliably with the parsetok.c assertion failure, but can be made to pass with a smaller ASAN redzone. The redzone must be a power of 2 and at least 16, default of 2048.

Fails:
ASAN_OPTIONS="max_redzone=256" ./python Tools/scripts/run_tests.py test_pydoc

Passes:
ASAN_OPTIONS="max_redzone=128" ./python Tools/scripts/run_tests.py test_pydoc

Values of 16, 32, and 64 also pass.

@Ned, I have found a solution on Linux, will try on this evening with my
macOS laptop. 
I hope to come with a solution on this evening or tomorrow, but need to
check my proto.

New changeset 4cb525a1f0cba152d1da61e531f2ad5db633120a by Victor Stinner in branch 'master':
bpo-36356: pymain_exit_error() only call pymain_free() for exit (GH-12968)
https://github.com/python/cpython/commit/4cb525a1f0cba152d1da61e531f2ad5db633120a

See also bpo-36724 "Clear _PyRuntime at exit".

New changeset b36e5d627d4232a01850707eb78a5067f3fd77f4 by Victor Stinner in branch 'master':
bpo-36356: Destroy the GIL at exit (GH-12453)
https://github.com/python/cpython/commit/b36e5d627d4232a01850707eb78a5067f3fd77f4

The initial issue has been fixed, so I close the issue. I prefer to get more specific issues like bpo-36724 "Clear _PyRuntime at exit" to following changes.

Thanks Ben Harper for the bug report and the fix.

Note: Ben, I like your songs! Good to know that you chose to contribute to Python in your free time! ;-)

Oh. test_daemon_threads_shutdown_stdout_deadlock of test_io fails randomly on buildbots.

I ran "./python -m test test_io -m '*daemon*' -F -j4" for 5 minutes on my Fedora 29 before merging my PR, but I didn't get any failure :-(

======================================================================
FAIL: test_daemon_threads_shutdown_stdout_deadlock (test.test_io.CMiscIOTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4186, in test_daemon_threads_shutdown_stdout_deadlock
    self.check_daemon_threads_shutdown_deadlock('stdout')
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4177, in check_daemon_threads_shutdown_deadlock
    self.assertIn("Fatal Python error: could not acquire lock "
AssertionError: "Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads" not found in 'Fatal Python error: PyCOND_FINI(_PyRuntime.ceval.gil.cond) failed'

Failures:

AMD64 Fedora Rawhide Clang Installed 3.x: fail then pass
https://buildbot.python.org/all/#/builders/188/builds/93

s390x Debian 3.x: fail then pass
https://buildbot.python.org/all/#/builders/13/builds/2799

s390x RHEL 3.x: fail twice
https://buildbot.python.org/all/#/builders/21/builds/2780

AMD64 Debian root 3.x: fail twice
https://buildbot.python.org/all/#/builders/27/builds/2759

New changeset 99fcc616d400cd31af0733c3f8cc93bcc1d32a44 by Victor Stinner in branch 'master':
Revert "bpo-36356: Destroy the GIL at exit (GH-12453)" (GH613006)
https://github.com/python/cpython/commit/99fcc616d400cd31af0733c3f8cc93bcc1d32a44

I just reverted the change. I'm unable to reproduce and I am not available this week to investigate anyway.

New changeset 321def805abc5b7c92c7e90ca90cb2434fdab855 by Yury Selivanov (Ben Harper) in branch 'master':
bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)
https://github.com/python/cpython/commit/321def805abc5b7c92c7e90ca90cb2434fdab855

New changeset 13915a3100608f011b29da2f3716c990f523b631 by Miss Islington (bot) in branch '3.8':
bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)
https://github.com/python/cpython/commit/13915a3100608f011b29da2f3716c990f523b631

Can we close this issue, or is there a remaining known bug?

FWIW, the assertion error on macOS reported above in msg338774 does not seem to occur anymore with current HEAD of master and with recent system clang (Apple clang version 11.0.0 (clang-1100.0.33.8)) on macOS 10.14.6.

The default build of python with ASAN builds successfully. Test suite and PGO build are currently blocked, but that can be separated if it makes more sense to handle them individually.

Would it make sense to add an ASAN build to the CI pipeline to make detection automatic going forward?

> The default build of python with ASAN builds successfully. Test suite and PGO build are currently blocked, but that can be separated if it makes more sense to handle them individually.

Yeah, please open one issue per ASAN issue (try to group similar warnings in the same Python issue?).


> Would it make sense to add an ASAN build to the CI pipeline to make detection automatic going forward?

Gregory maintains a clang UBSan buildbot:
https://buildbot.python.org/all/#/builders/135

Usually I prefer to setup a buildbot when most issues are fixed.

I close this issue.