classification
Title: AddressSanitizer: bad-free - hello world c extension
Type: Stage: resolved
Components: Windows Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: FrancescElies, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2021-11-19 12:06 by FrancescElies, last changed 2021-11-30 07:12 by FrancescElies. This issue is now closed.

Messages (3)
msg406578 - (view) Author: Francesc Elies (FrancescElies) Date: 2021-11-19 12:06
Hi,

Context
=======
we are compiling a dll with clang and it's address sanitizer and loading it via cffi, at random spots ASAN complains with bad-free.

      ==15100==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x01991a850310 in thread T0
          #0 0x7ffa1dcc7f31  (C:\LLVM-13.0.0-win64\lib\clang\13.0.0\lib\windows\clang_rt.asan_dynamic-x86_64.dll+0x180037f31)
          #1 0x7ffa3aea59ec in _PyObject_Realloc D:\_w\1\s\Objects\obmalloc.c:2011
          #2 0x7ffa3af7f347 in _PyObject_GC_Resize D:\_w\1\s\Modules\gcmodule.c:2309
          #3 0x7ffa3aedeeaa in _PyEval_EvalCode D:\_w\1\s\Python\ceval.c:4101
          ...
          See links below for a full trace

The project where we see this it's quite complex and with many moving parts therefore we made a minimal reproducible example.

Reprex
======

The test boils down to the following:

    win32api.LoadLibrary("LLVM-13.0.0-win64/lib/clang/13.0.0/lib/windows/clang_rt.asan_dynamic-x86_64.dll")
    import hello  # hello is our compiled c extension with clang and ASAN
    print(hello.system())
    import pdb


If if comment the last line (import pdb) ASAN does not complain.
If instead of import pdb you import numpy the same problem can be seen.

See the following failing build https://github.com/FrancescElies/min_reprex_python_c_extension_asan/runs/4263693010?check_suite_focus=true
See here the CI test https://github.com/FrancescElies/min_reprex_python_c_extension_asan/blob/d966d3a472df71977dc6519a76be0120d2d58d39/test.py


We did not try this in linux yet, would that help?

Is this a bug? Or are we doing something wrong?

Thanks in advance for your time.
msg406979 - (view) Author: Francesc Elies (FrancescElies) Date: 2021-11-25 08:16
I tested the same script in ubuntu and got the following.

==1361==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.

While on windows he does not complain about ASan runtime not bein first in initial library list.

Once I use LD_PRELOAD in linux. Lsan detected memory leaks, after suppressing them with LSAN_OPTIONS=detect_leaks=0, the script ran fine.

So far I could not find an LD_PRELOAD equivalent in windows.

I am suspecting if one would compile python with asan and run the test on windows, I would get similar results as in linux. Sadly at the moment asan suppressions doesn't seem to work on windows.

I think this is not a real bug on python but the way we are loading asan-rt in windows. Once I am sure I will colse this bug.
msg407354 - (view) Author: Francesc Elies (FrancescElies) Date: 2021-11-30 07:12
I am closing this one, bad-free is in this case a false positive.

Starting python and loading a dll which was linked with asan is incorrect.

One should asan-rt as earyly as possible, in order to do that in linux one should use LD_PRELOAD but in windows it's a bit more convoluted.

We made an executable which loads asan and instructed the dll DLLs to use the runtime linked into the main executable to avoid shadow memory collisions.

In the official docs https://clang.llvm.org/docs/AddressSanitizer.html I could not find much about asan with clang under windows. 

Despite the library file names where not the same as in llvm master and that the blogpost has msvc in mind instead of clang this blog https://devblogs.microsoft.com/cppblog/addresssanitizer-asan-for-windows-with-msvc/#contributions-to-asan-runtime pointed us in the right direction.
History
Date User Action Args
2021-11-30 07:12:30FrancescEliessetstatus: open -> closed
resolution: not a bug
messages: + msg407354

stage: resolved
2021-11-25 08:16:11FrancescEliessetmessages: + msg406979
2021-11-19 12:06:17FrancescEliescreate