Address sanitizer (gcc version) is generating false positives #88529
Comments
|
Running the buildbot for #26595 The tests pass with an added assert to verify that the address in question is in bounds. All failures seem to happen after a fork, which seems to be a longstanding weakness of the address sanitizer. I'd like to merge #26595. I'd also like to keep the buildbots working. Dichotomy. |
|
Buildbots have the priority, please don't merge anything that breaks the buildbots. We should figure out how to silence the false positive if indeed is one, but I don't think it is. The address sanitizer is not speculative, it literally checks if someone has written out of bounds |
|
I can reproduce that failure of address sanitizer with GCC and clang separately, so it doesn't seem like a false positive. |
|
Also, I can reproduce locally without any fork whatsoever: ./python -m test test_lib2to3 SUMMARY: AddressSanitizer: heap-buffer-overflow Python/ceval.c:3549 in _PyEval_EvalFrameDefault |
|
Even valgrind sees the problem: ❯ valgrind ./python -m test test_lib2to3 |
|
I'm closing this as 3 different tools complain about the same thing, with a very simple reproducer. |
|
If I run the following on main I get 22 failures. So something is broken. test_lib2to3 does fork; at least it does when I run it under gdb. |
|
As I mentioned in https://bugs.python.org/issue43693, the way to run it is the following: $ export ASAN_OPTIONS=detect_leaks=0:allocator_may_return_null=1:handle_segv=0
$ ./configure --with-address-sanitizer --without-pymalloc
$ make -j -s
$ ./python -m test test_lib2to3There are known failures with things like ctypes and libcript so the following tests are ignored: test_ctypes test_capi test_crypt test_decimal test_faulthandler test_interpreters ----------------- test_lib2to3 does one fork for this call: https://github.com/python/cpython/blob/main/Lib/lib2to3/tests/test_parser.py#L64 but that doesn't matter. Comment that and keeps falining |
|
I still get quite a few failures on the main branch. There isn't anything wrong with that line, but as I plan to change it anyway I guess it doesn't matter: I plan to get the main branch passing the tests with address sanitization (at least on my machine with my gcc :) Running the tests doesn't seem to take that long (at least not compared with refleak tests). |
markshannon
changed the title
markshannon
changed the title
That's a good idea,let me see how easy doing that would be |
Even if you don't trust the sanitizer, you can also use valgrind, it shows the same error: ❯ valgrind ./python -m test test_lib2to3 For running valgrind, make sure do disable pymalloc: export PYTHONMALLOC=malloc |
|
This is not a false positive. I break into gdb at the moment the sanitizer makes the report and inspecting the values that apparently are wrong. I did that by breaking into __sanitizer::ColorizeReports which is called for making the report. THis is what I did: $export CFLAGS='-g3 -O0' Breakpoint 1, __sanitizer::ColorizeReports () That ep->me_key is obviously corrupted. |
|
What commit are you running that on? |
|
commit e858ea1 (HEAD -> specialize-load-attr) diff --git a/Python/ceval.c b/Python/ceval.c
index 06a02b40f9..784d0244e8 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -3517,6 +3517,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR);
PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict;
DEOPT_IF(dict->ma_keys->dk_version != cache1->dk_version_or_hint, LOAD_ATTR);
+ assert(cache0->index < dict->ma_keys->dk_nentries);
PyDictKeyEntry *ep = DK_ENTRIES(dict->ma_keys) + cache0->index;
res = ep->me_value;
DEOPT_IF(res == NULL, LOAD_ATTR); |
|
This line seems to be responsible for most of the failures: Which does appear to be a true positive. |
|
From the gdb session looks to me that you are reading freed or invalid memory from the dictionary keys. Well, I'm quite sure you are doing that, but I don't know why they is happening. |
|
It looks like I've been a bit unfair to the address sanitizer. It does appear to produce incorrect locations sometimes, but that's not really a false positive and the reports are generally useful. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: