classification
Title: Support building Python with Clang sanitizer rules
Type: enhancement Stage: resolved
Components: Build Versions: Python 3.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Jeffrey.Walton, benjamin.peterson, brett.cannon, ned.deily
Priority: normal Keywords:

Created on 2014-03-15 16:14 by Jeffrey.Walton, last changed 2014-03-23 13:15 by neologix. This issue is now closed.

Repositories containing patches
http://hg.python.org/cpython
Messages (8)
msg213661 - (view) Author: Jeffrey Walton (Jeffrey.Walton) * Date: 2014-03-15 16:14
From Python head in mercurial.

When building Python under Clang's sanitizers, we provide a couple of flags to instrument binaries with the sanitizers. For example:

export CC=/usr/local/bin/clang
export CXX=/usr/local/bin/clang++
export CFLAGS="-g3 -fsanitize=undefined -fsanitize=address"
export CXXFLAGS="-g3 -fsanitize=undefined -fsanitize=address -fno-sanitize=vptr"
./configure
make

However, `make` will fail due to some missing sanitizer libraries. The libraries are added at the link stage by Clang, but the invocation must include the -fsanitize=... flags.

The recipe for $(BUILDPYTHON) in the Makefile does not include necessary CFLAGS:

# Build the interpreter
$(BUILDPYTHON):	Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
	$(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)

The result is a failed link when building with the sanitizers.

It would be great if the sanizter flags (-fsanitize=undefined -fsanitize=address -fno-sanitize=vptr) were cherry picked from the FLAGS by the build system and added to the recipe as required:

$(BUILDPYTHON):	Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
	$(LINKCC) -fsanitize=undefined -fsanitize=address -fno-sanitize=vptr $(PY_LDFLAGS) $(LINKFORSHARED) ...

Please consider picking up the sanitizer flags and adding them to the build rule.
msg213662 - (view) Author: Jeffrey Walton (Jeffrey.Walton) * Date: 2014-03-15 16:21
And:

Modules/_testembed: Modules/_testembed.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
	$(LINKCC) -g3 -fsanitize=address $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/_testembed.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
msg213663 - (view) Author: Jeffrey Walton (Jeffrey.Walton) * Date: 2014-03-15 16:24
And:

Modules/_freeze_importlib: Modules/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN)
	$(LINKCC) -g3 -fsanitize=address $(PY_LDFLAGS) -o $@ Modules/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
msg213673 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-03-15 19:11
("Cherry pick" has a special meaning for the release process.)
msg213689 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-03-15 22:34
Just use LDFLAGS.
msg213690 - (view) Author: Jeffrey Walton (Jeffrey.Walton) * Date: 2014-03-15 22:55
On Sat, Mar 15, 2014 at 6:34 PM, Benjamin Peterson
<report@bugs.python.org> wrote:
>
> Benjamin Peterson added the comment:
>
> Just use LDFLAGS.
Yeah, I tried that and broke the sanitizer:
https://groups.google.com/d/msg/address-sanitizer/cu2WoD1Bwx8/zUoY9GH7oHkJ.

The other fallback is to stuff it into CC and CXX. But that breaks
some autotool projects. (Python is OK, though).

Jeff
msg213691 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-03-15 23:11
CFLAGS="-g3 -fsanitize=address" LDFLAGS="-fsanitize=address" ./configure --with-system-expat && make -j4

works for me.
msg213693 - (view) Author: Jeffrey Walton (Jeffrey.Walton) * Date: 2014-03-15 23:41
On Sat, Mar 15, 2014 at 7:11 PM, Benjamin Peterson
<report@bugs.python.org> wrote:
>
> Benjamin Peterson added the comment:
>
> CFLAGS="-g3 -fsanitize=address" LDFLAGS="-fsanitize=address" ./configure --with-system-expat && make -j4
>
> works for me.
Oh,my bad. I thought you meant add the libraries like
libclang_rt.asan_osx.a to the flags.

Jeff
History
Date User Action Args
2014-03-23 13:15:58neologixsetstatus: open -> closed
resolution: not a bug
stage: resolved
2014-03-15 23:41:13Jeffrey.Waltonsetmessages: + msg213693
2014-03-15 23:11:41benjamin.petersonsetmessages: + msg213691
2014-03-15 22:55:44Jeffrey.Waltonsetmessages: + msg213690
2014-03-15 22:34:01benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg213689
2014-03-15 19:11:18ned.deilysetnosy: + ned.deily

messages: + msg213673
title: Cherry pick CFLAGS, add to flags for $(BUILDPYTHON) Makefile rule -> Support building Python with Clang sanitizer rules
2014-03-15 18:44:34brett.cannonsetnosy: + brett.cannon
2014-03-15 16:24:03Jeffrey.Waltonsetmessages: + msg213663
2014-03-15 16:21:17Jeffrey.Waltonsetmessages: + msg213662
2014-03-15 16:14:35Jeffrey.Waltoncreate