➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2016-08-16 22:18 by geeknik, last changed 2022-04-11 14:58 by admin. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 7869 | merged | benjamin.peterson, 2018-06-23 03:34 | |
| PR 7915 | merged | zach.ware, 2018-06-25 21:48 | |
| Messages (9) | |||
|---|---|---|---|
| msg272889 - (view) | Author: geeknik (geeknik) | Date: 2016-08-16 22:18 | |
CC=afl-clang-fast CXX=afl-clang-fast++ AFL_USE_ASAN=1 ./configure --disable-ipv6 Passes fine. AFL_USE_ASAN=1 make Ends with the following: Parser/pgen ./Grammar/Grammar Include/graminit.h Python/graminit.c ================================================================= ==29392==ERROR: LeakSanitizer: detected memory leaks Direct leak of 12416 byte(s) in 8 object(s) allocated from: #0 0x4a1f2e in realloc (/root/cpython/Parser/pgen+0x4a1f2e) #1 0x4cfcca in PyMem_RawRealloc /root/cpython/Objects/obmalloc.c:414:12 Indirect leak of 2080 byte(s) in 3 object(s) allocated from: #0 0x4a1f2e in realloc (/root/cpython/Parser/pgen+0x4a1f2e) #1 0x4cfcca in PyMem_RawRealloc /root/cpython/Objects/obmalloc.c:414:12 Indirect leak of 898 byte(s) in 86 object(s) allocated from: #0 0x4a1c3b in __interceptor_malloc (/root/cpython/Parser/pgen+0x4a1c3b) #1 0x7fbd7abe3989 in __strdup /build/glibc-uPj9cH/glibc-2.19/string/strdup.c:42 Indirect leak of 520 byte(s) in 1 object(s) allocated from: #0 0x4a1c3b in __interceptor_malloc (/root/cpython/Parser/pgen+0x4a1c3b) #1 0x4d3fa0 in PyMem_RawMalloc /root/cpython/Objects/obmalloc.c:396:12 #2 0x4d3fa0 in _PyObject_Alloc /root/cpython/Objects/obmalloc.c:1467 Indirect leak of 178 byte(s) in 33 object(s) allocated from: #0 0x4a1c3b in __interceptor_malloc (/root/cpython/Parser/pgen+0x4a1c3b) #1 0x4c6756 in translabel /root/cpython/Parser/grammar.c:197:28 #2 0x4c6756 in _Py_translatelabels /root/cpython/Parser/grammar.c:141 SUMMARY: AddressSanitizer: 16092 byte(s) leaked in 131 allocation(s). Makefile:804: recipe for target 'Include/graminit.h' failed make: *** [Include/graminit.h] Error 23 |
|||
| msg272996 - (view) | Author: Ned Deily (ned.deily) *  |
Date: 2016-08-17 22:33 | |
Thanks for the report but, AFAIK, pgen is only used during the build of Python and pgen is not installed (by "make install"). This doesn't seem like it is worth worrying about. Or am I missing something? Feel free to reopen if so or if someone comes up with a patch. |
|||
| msg272998 - (view) | Author: geeknik (geeknik) | Date: 2016-08-17 22:37 | |
FYI, I was only able to build Python with ASAN by passing ASAN_OPTIONS=detect_leaks=0 along with the make command, otherwise ASAN wanted to stop the build process as soon as it detected this leak. |
|||
| msg273000 - (view) | Author: Ned Deily (ned.deily) * |
Date: 2016-08-17 22:52 | |
OK, that's not unreasonable and I see there have been earlier issues opened and addressed for similar problems (e.g. Issue18695). Perhaps someone will want to dive in. |
|||
| msg273268 - (view) | Author: Xiang Zhang (xiang.zhang) * |
Date: 2016-08-21 05:48 | |
Valgrind can also complain: valgrind --leak-check=yes --suppressions=Misc/valgrind-python.supp Parser/pgen ./Grammar/Grammar Include/graminit.h Python/graminit.c ==6836== Memcheck, a memory error detector ==6836== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==6836== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==6836== Command: Parser/pgen ./Grammar/Grammar Include/graminit.h Python/graminit.c ==6836== Translating labels ... ==6836== ==6836== HEAP SUMMARY: ==6836== in use at exit: 18,236 bytes in 133 blocks ==6836== total heap usage: 848 allocs, 715 frees, 781,340 bytes allocated ==6836== ==6836== 6 bytes in 1 blocks are possibly lost in loss record 1 of 11 ==6836== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x50E1379: strdup (strdup.c:42) ==6836== by 0x403405: _Py_addlabel (grammar.c:104) ==6836== by 0x403D40: newnfagrammar (pgen.c:116) ==6836== by 0x403E21: metacompile (pgen.c:158) ==6836== by 0x4056F1: _Py_pgen (pgen.c:668) ==6836== by 0x40CBE3: getgrammar (pgenmain.c:125) ==6836== by 0x40C932: main (pgenmain.c:64) ==6836== ==6836== 178 bytes in 33 blocks are possibly lost in loss record 2 of 11 ==6836== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x403838: translabel (grammar.c:203) ==6836== by 0x40356A: _Py_translatelabels (grammar.c:147) ==6836== by 0x405711: _Py_pgen (pgen.c:670) ==6836== by 0x40CBE3: getgrammar (pgenmain.c:125) ==6836== by 0x40C932: main (pgenmain.c:64) ==6836== ==6836== 584 bytes in 1 blocks are possibly lost in loss record 3 of 11 ==6836== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x405774: _PyMem_RawMalloc (obmalloc.c:83) ==6836== by 0x4076DE: _PyMem_DebugRawAlloc (obmalloc.c:1903) ==6836== by 0x4077CC: _PyMem_DebugRawMalloc (obmalloc.c:1926) ==6836== by 0x405EDF: PyMem_RawMalloc (obmalloc.c:396) ==6836== by 0x406D16: _PyObject_Alloc (obmalloc.c:1466) ==6836== by 0x407515: _PyObject_Realloc (obmalloc.c:1756) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x407BB2: _PyMem_DebugRealloc (obmalloc.c:2049) ==6836== by 0x40620A: PyObject_Realloc (obmalloc.c:507) ==6836== by 0x4030C6: _Py_addstate (grammar.c:55) ==6836== by 0x405497: convert (pgen.c:611) ==6836== ==6836== 640 bytes in 1 blocks are possibly lost in loss record 4 of 11 ==6836== at 0x4C2DD9F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x4057F3: _PyMem_RawRealloc (obmalloc.c:105) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x405F70: PyMem_RawRealloc (obmalloc.c:414) ==6836== by 0x40756A: _PyObject_Realloc (obmalloc.c:1777) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x407BB2: _PyMem_DebugRealloc (obmalloc.c:2049) ==6836== by 0x40620A: PyObject_Realloc (obmalloc.c:507) ==6836== by 0x403B2A: addnfastate (pgen.c:52) ==6836== by 0x4047E5: compile_atom (pgen.c:300) ==6836== by 0x4045E9: compile_item (pgen.c:268) ==6836== by 0x40438D: compile_alt (pgen.c:237) ==6836== ==6836== 744 bytes in 1 blocks are definitely lost in loss record 5 of 11 ==6836== at 0x4C2DD9F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x4057F3: _PyMem_RawRealloc (obmalloc.c:105) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x405F70: PyMem_RawRealloc (obmalloc.c:414) ==6836== by 0x40756A: _PyObject_Realloc (obmalloc.c:1777) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x407BB2: _PyMem_DebugRealloc (obmalloc.c:2049) ==6836== by 0x40620A: PyObject_Realloc (obmalloc.c:507) ==6836== by 0x403D8C: addnfa (pgen.c:126) ==6836== by 0x403F4E: compile_rule (pgen.c:178) ==6836== by 0x403E86: metacompile (pgen.c:164) ==6836== by 0x4056F1: _Py_pgen (pgen.c:668) ==6836== ==6836== 800 bytes in 1 blocks are possibly lost in loss record 6 of 11 ==6836== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x405774: _PyMem_RawMalloc (obmalloc.c:83) ==6836== by 0x4076DE: _PyMem_DebugRawAlloc (obmalloc.c:1903) ==6836== by 0x407914: _PyMem_DebugRawRealloc (obmalloc.c:1971) ==6836== by 0x405F70: PyMem_RawRealloc (obmalloc.c:414) ==6836== by 0x40631C: new_arena (obmalloc.c:1064) ==6836== by 0x4067FC: _PyObject_Alloc (obmalloc.c:1339) ==6836== by 0x406D66: _PyObject_Malloc (obmalloc.c:1476) ==6836== by 0x4076DE: _PyMem_DebugRawAlloc (obmalloc.c:1903) ==6836== by 0x4077CC: _PyMem_DebugRawMalloc (obmalloc.c:1926) ==6836== by 0x407B23: _PyMem_DebugMalloc (obmalloc.c:2028) ==6836== by 0x406179: PyObject_Malloc (obmalloc.c:489) ==6836== ==6836== 892 bytes in 85 blocks are possibly lost in loss record 7 of 11 ==6836== at 0x4C2BBCF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x50E1379: strdup (strdup.c:42) ==6836== by 0x40304A: _Py_adddfa (grammar.c:42) ==6836== by 0x405691: maketables (pgen.c:655) ==6836== by 0x405701: _Py_pgen (pgen.c:669) ==6836== by 0x40CBE3: getgrammar (pgenmain.c:125) ==6836== by 0x40C932: main (pgenmain.c:64) ==6836== ==6836== 2,272 bytes in 3 blocks are possibly lost in loss record 8 of 11 ==6836== at 0x4C2DD9F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x4057F3: _PyMem_RawRealloc (obmalloc.c:105) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x405F70: PyMem_RawRealloc (obmalloc.c:414) ==6836== by 0x40756A: _PyObject_Realloc (obmalloc.c:1777) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x407BB2: _PyMem_DebugRealloc (obmalloc.c:2049) ==6836== by 0x40620A: PyObject_Realloc (obmalloc.c:507) ==6836== by 0x4030C6: _Py_addstate (grammar.c:55) ==6836== by 0x405497: convert (pgen.c:611) ==6836== by 0x404F7B: makedfa (pgen.c:489) ==6836== by 0x4056C0: maketables (pgen.c:656) ==6836== ==6836== 2,864 bytes in 1 blocks are possibly lost in loss record 9 of 11 ==6836== at 0x4C2DD9F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x4057F3: _PyMem_RawRealloc (obmalloc.c:105) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x405F70: PyMem_RawRealloc (obmalloc.c:414) ==6836== by 0x40756A: _PyObject_Realloc (obmalloc.c:1777) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x407BB2: _PyMem_DebugRealloc (obmalloc.c:2049) ==6836== by 0x40620A: PyObject_Realloc (obmalloc.c:507) ==6836== by 0x4033AA: _Py_addlabel (grammar.c:98) ==6836== by 0x40480D: compile_atom (pgen.c:301) ==6836== by 0x4045E9: compile_item (pgen.c:268) ==6836== by 0x40433D: compile_alt (pgen.c:232) ==6836== ==6836== 3,464 bytes in 1 blocks are possibly lost in loss record 10 of 11 ==6836== at 0x4C2DD9F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x4057F3: _PyMem_RawRealloc (obmalloc.c:105) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x405F70: PyMem_RawRealloc (obmalloc.c:414) ==6836== by 0x40756A: _PyObject_Realloc (obmalloc.c:1777) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x407BB2: _PyMem_DebugRealloc (obmalloc.c:2049) ==6836== by 0x40620A: PyObject_Realloc (obmalloc.c:507) ==6836== by 0x402FE4: _Py_adddfa (grammar.c:36) ==6836== by 0x405691: maketables (pgen.c:655) ==6836== by 0x405701: _Py_pgen (pgen.c:669) ==6836== by 0x40CBE3: getgrammar (pgenmain.c:125) ==6836== ==6836== 5,792 bytes in 5 blocks are possibly lost in loss record 11 of 11 ==6836== at 0x4C2DD9F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==6836== by 0x4057F3: _PyMem_RawRealloc (obmalloc.c:105) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x405F70: PyMem_RawRealloc (obmalloc.c:414) ==6836== by 0x40756A: _PyObject_Realloc (obmalloc.c:1777) ==6836== by 0x407999: _PyMem_DebugRawRealloc (obmalloc.c:1986) ==6836== by 0x407BB2: _PyMem_DebugRealloc (obmalloc.c:2049) ==6836== by 0x40620A: PyObject_Realloc (obmalloc.c:507) ==6836== by 0x403B2A: addnfastate (pgen.c:52) ==6836== by 0x4047E5: compile_atom (pgen.c:300) ==6836== by 0x4045E9: compile_item (pgen.c:268) ==6836== by 0x40433D: compile_alt (pgen.c:232) ==6836== ==6836== LEAK SUMMARY: ==6836== definitely lost: 744 bytes in 1 blocks ==6836== indirectly lost: 0 bytes in 0 blocks ==6836== possibly lost: 17,492 bytes in 132 blocks ==6836== still reachable: 0 bytes in 0 blocks ==6836== suppressed: 0 bytes in 0 blocks ==6836== ==6836== For counts of detected and suppressed errors, rerun with: -v ==6836== ERROR SUMMARY: 11 errors from 11 contexts (suppressed: 484 from 2) |
|||
| msg320228 - (view) | Author: Zackery Spytz (ZackerySpytz) *  |
Date: 2018-06-22 12:16 | |
These memory leaks were fixed in commit 9ac11a752a19c3b8607582a3d5ccb615c467124b, though the fix wasn't applied to 2.7. |
|||
| msg320273 - (view) | Author: Ned Deily (ned.deily) * |
Date: 2018-06-22 21:16 | |
Thanks, Zackery. @Benjamin, any opinion about a need for a 2.7 backport? Otherwise, it sounds like this can be closed. |
|||
| msg320296 - (view) | Author: Benjamin Peterson (benjamin.peterson) * |
Date: 2018-06-23 04:04 | |
New changeset 482fac74999028af4764645a1f28dd4289b2f255 by Benjamin Peterson in branch '2.7': [2.7] properly free memory in pgen. (GH-7869) (closes bpo-27780 https://github.com/python/cpython/commit/482fac74999028af4764645a1f28dd4289b2f255 |
|||
| msg320467 - (view) | Author: Benjamin Peterson (benjamin.peterson) * |
Date: 2018-06-26 03:40 | |
New changeset 16137fe22f3f9bbbd50fac2a729172976c553718 by Benjamin Peterson (Zachary Ware) in branch '2.7': bpo-27780: Make pgen.c C89 compliant (GH-7915) https://github.com/python/cpython/commit/16137fe22f3f9bbbd50fac2a729172976c553718 |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:34 | admin | set | github: 71967 |
| 2018-06-26 03:40:22 | benjamin.peterson | set | messages: + msg320467 |
| 2018-06-25 21:48:49 | zach.ware | set | pull_requests: + pull_request7518 |
| 2018-06-23 04:04:24 | benjamin.peterson | set | status: open -> closed resolution: fixed messages: + msg320296 stage: patch review -> resolved |
| 2018-06-23 03:34:21 | benjamin.peterson | set | keywords:
+ patch stage: needs patch -> patch review pull_requests: + pull_request7477 |
| 2018-06-22 21:16:20 | ned.deily | set | nosy:
+ benjamin.peterson messages: + msg320273 |
| 2018-06-22 12:16:36 | ZackerySpytz | set | nosy:
+ ZackerySpytz messages: + msg320228 |
| 2016-11-05 14:56:14 | r.david.murray | link | issue28620 superseder |
| 2016-08-21 05:48:09 | xiang.zhang | set | nosy:
+ xiang.zhang messages: + msg273268 |
| 2016-08-17 22:52:41 | ned.deily | set | status: closed -> open versions: + Python 3.6 title: Memory leak during Python build (from git c3ff7e7) on Debian 8.5 x64 -> memory leaks in pgen build step abort build with address sanitizer enabled messages: + msg273000 resolution: wont fix -> (no value) stage: resolved -> needs patch |
| 2016-08-17 22:37:58 | geeknik | set | messages: + msg272998 |
| 2016-08-17 22:33:52 | ned.deily | set | status: open -> closed priority: normal -> low nosy: + ned.deily messages: + msg272996 resolution: wont fix stage: resolved |
| 2016-08-16 22:18:56 | geeknik | create | |