Python implements debug hooks on Python memory allocators:
https://docs.python.org/dev/c-api/memory.html#c.PyMem_SetupDebugHooks

Problem: buffer understand and buffer overflow are only detected when a memory block is released, which may occur far from the code responsible to create the buffer.

Attached patch dumps the traceback where a memory block was allocated on error in malloc debug hooks. The feature requires to enable tracemalloc to record tracebacks.


test.py used in below example:
----
import _testcapi

def f():
    _testcapi.pymem_buffer_overflow()

f()
----


Example:
---
haypo@selma$ ./python -X tracemalloc=5 test.py 
Debug memory block at address p=0x22e8be0: API 'm'
    16 bytes originally requested
    The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
    The 8 pad bytes at tail=0x22e8bf0 are not all FORBIDDENBYTE (0xfb):
        at tail+0: 0x78 *** OUCH
        ...
    The block was made by call #37240 to debug malloc/realloc.
    Data at p: cb cb cb cb cb cb cb cb cb cb cb cb cb cb cb cb

Memory block traceback (most recent call first):
  File "test.py", line 4
  File "test.py", line 6

Fatal Python error: bad trailing pad byte

Current thread 0x00007f4a93d9c700 (most recent call first):
  File "test.py", line 4 in f
  File "test.py", line 6 in <module>
Abandon (core dumped)
---

The patch adds the "Memory block traceback" section.

See also the issue #26567: "Use tracemalloc to display the traceback where an object was allocated when a ResourceWarning is emitted".

I reviewed my own patch on Rietveld :-)

New changeset 18a19e62bac5 by Victor Stinner in branch 'default':
Enhance and rewrite traceback dump C functions
https://hg.python.org/cpython/rev/18a19e62bac5

New changeset fea3c6e9a38e by Victor Stinner in branch '3.5':
_tracemalloc: store lineno as unsigned int
https://hg.python.org/cpython/rev/fea3c6e9a38e

New changeset cef6a32d805f by Victor Stinner in branch 'default':
On memory error, dump the memory block traceback
https://hg.python.org/cpython/rev/cef6a32d805f

New changeset 8215dae7ec3c by Victor Stinner in branch 'default':
Oops, revert unwanted change used to create an example
https://hg.python.org/cpython/rev/8215dae7ec3c

New changeset 769dfcb701ee by Victor Stinner in branch 'default':
Issue #26564: Fix test_capi
https://hg.python.org/cpython/rev/769dfcb701ee

FYI Victor if you haven’t already noticed, the Windows buildbots are choking on a variable-length array:

..\Python\traceback.c(513): error C2057: expected constant expression [D:\buildarea\3.x.bolen-windows10\build\PCbuild\pythoncore.vcxproj]
..\Python\traceback.c(513): error C2466: cannot allocate an array of constant size 0 [D:\buildarea\3.x.bolen-windows10\build\PCbuild\pythoncore.vcxproj]
..\Python\traceback.c(513): error C2133: 'buffer': unknown size [D:\buildarea\3.x.bolen-windows10\build\PCbuild\pythoncore.vcxproj]

New changeset 5f2284ecf9c6 by Victor Stinner in branch 'default':
Fix compilation error of traceback.c on Windows
https://hg.python.org/cpython/rev/5f2284ecf9c6

Martin Panter added the comment:
> FYI Victor if you haven’t already noticed, the Windows buildbots are choking on a variable-length array:

Thank you, sometimes I miss buildbot failures.

I pushed changes which had 2 tests, so I knew that all buildbots were
red, and I planned to come back the day after to check.

For Windows, it's a pity, the expression is constant. Stupid compiler
:-) VS2015 doesn't support C99 yet? Anyway, it's trivial to fix the
code. I just fixed it.

I will check again if buildbots are now happy with the current code ;-)

Ok, buildbots are green again. I close the issue.

New changeset 7b079adb0774 by Victor Stinner in branch 'default':
Enhance documentation on malloc debug hooks
https://hg.python.org/cpython/rev/7b079adb0774

I'm reopening this to commit a fix for a comment which became obsolete following this change: https://hg.python.org/cpython/rev/18a19e62bac5.

New changeset 40125ab3252453bf205ed906e46bf9741c27bf9d by Irit Katriel in branch 'master':
bpo-26564: fix obsolete comment in traceback.c (GH-23819)
https://github.com/python/cpython/commit/40125ab3252453bf205ed906e46bf9741c27bf9d

New changeset cecbaa3a80d5ae28cdd4129d6d2c4395c12a89e4 by Miss Islington (bot) in branch '3.8':
bpo-26564: fix obsolete comment in traceback.c (GH-23819)
https://github.com/python/cpython/commit/cecbaa3a80d5ae28cdd4129d6d2c4395c12a89e4

New changeset 1c70d40e94741578ce28b8851fb65372ac2e7942 by Miss Islington (bot) in branch '3.9':
bpo-26564: fix obsolete comment in traceback.c (GH-23819)
https://github.com/python/cpython/commit/1c70d40e94741578ce28b8851fb65372ac2e7942

I am wondering why 53/22? 5/2 is as good for 8-bytes long -- both gives 21-bytes buffer.

I do not propose to change anything. It is just curiosity.

> I am wondering why 53/22? 5/2 is as good for 8-bytes long -- both gives 21-bytes buffer.

53/22 is closer to math.log(256)/math.log(10) than 5/2, and 8*53 should not overflow :-)