classification
Title: print ResourceWarning object traceback when raised as error with -W error and -X tracemalloc
Type: Stage: resolved
Components: Versions:
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: graingert, gvanrossum, vstinner
Priority: normal Keywords:

Created on 2019-06-20 16:16 by graingert, last changed 2020-09-16 19:41 by gvanrossum. This issue is now closed.

Messages (3)
msg346129 - (view) Author: Thomas Grainger (graingert) * Date: 2019-06-20 16:16
Current behaviour 

$ cat foo.py
def foo():
    open('foo', 'wb')


foo()
$ python3.8 -W all::ResourceWarning -X tracemalloc=5 foo.py
foo.py:2: ResourceWarning: unclosed file <_io.BufferedWriter name='foo'>
  open('foo', 'wb')
Object allocated at (most recent call last):
  File "foo.py", lineno 5
    foo()
  File "foo.py", lineno 2
    open('foo', 'wb')
$ python3.8 -W error -X tracemalloc=5 foo.py
Exception ignored in: <_io.FileIO name='foo' mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.BufferedWriter name='foo'>

Expected behaviour

# Pastebin Eu0gvZ5Z
$ cat foo.py
def foo():
    open('foo', 'wb')


foo()
$ python3.8 -W all::ResourceWarning -X tracemalloc=5 foo.py
foo.py:2: ResourceWarning: unclosed file <_io.BufferedWriter name='foo'>
  open('foo', 'wb')
Object allocated at (most recent call last):
  File "foo.py", lineno 5
    foo()
  File "foo.py", lineno 2
    open('foo', 'wb')
$ python3.8 -W error -X tracemalloc=5 foo.py
Exception ignored in: <_io.FileIO name='foo' mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.BufferedWriter name='foo'>
  open('foo', 'wb')
Object allocated at (most recent call last):
  File "foo.py", lineno 5
    foo()
  File "foo.py", lineno 2
    open('foo', 'wb')
msg346130 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-20 16:22
You can implement this behavior using the new sys.unraisablehook (Python 3.8):
---
import sys
import tracemalloc

def foo():
    open('foo', 'wb')

def hook(unraisable):
    orig_unraisablehook(unraisable)
    if unraisable.object is not None:
        tb = tracemalloc.get_object_traceback(unraisable.object)
        if tb:
            print("Object allocated at:")
            for line in tb:
                print(line)


orig_unraisablehook = sys.unraisablehook
sys.unraisablehook = hook
tracemalloc.start(5)

foo()
---

Output:
---
Exception ignored in: <_io.FileIO name='foo' mode='wb' closefd=True>
Traceback (most recent call last):
  File "foo.py", line 5, in foo
    open('foo', 'wb')
ResourceWarning: unclosed file <_io.BufferedWriter name='foo'>
Object allocated at:
foo.py:21
foo.py:5
---

I understand that this issue is about modifying _PyErr_WriteUnraisableDefaultHook() in Python/errors.c (it's written in C) to implement the same logic than the default implementation of warnings._formatwarnmsg() when formatting a ResourceWarning with the source object:

        try:
            import tracemalloc
        # Logging a warning should not raise a new exception:
        # catch Exception, not only ImportError and RecursionError.
        except Exception:
            # don't suggest to enable tracemalloc if it's not available
            tracing = True
            tb = None
        else:
            tracing = tracemalloc.is_tracing()
            try:
                tb = tracemalloc.get_object_traceback(msg.source)
            except Exception:
                # When a warning is logged during Python shutdown, tracemalloc
                # and the import machinery don't work anymore
                tb = None

        if tb is not None:
            s += 'Object allocated at (most recent call last):\n'
            for frame in tb:
                s += ('  File "%s", lineno %s\n'
                      % (frame.filename, frame.lineno))

                try:
                    if linecache is not None:
                        line = linecache.getline(frame.filename, frame.lineno)
                    else:
                        line = None
                except Exception:
                    line = None
                if line:
                    line = line.strip()
                    s += '    %s\n' % line
        elif not tracing:
            s += (f'{category}: Enable tracemalloc to get the object '
                  f'allocation traceback\n')


In C, _PyTraceMalloc_GetTraceback() can be used to retrieve the traceback of an object saved by tracemalloc.

See also _PyMem_DumpTraceback() used to log the tracemaltoc when Python detects a buffer overflow:
https://pythondev.readthedocs.io/debug_tools.html#debug-memory-errors
msg377017 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-16 19:39
Closing for lack of activity.
History
Date User Action Args
2020-09-16 19:41:14gvanrossumsetstatus: open -> closed
stage: resolved
2020-09-16 19:39:12gvanrossumsetnosy: + gvanrossum
messages: + msg377017
2019-06-20 16:22:50vstinnersetnosy: + vstinner
messages: + msg346130
2019-06-20 16:18:24graingertsettitle: print ResourceWarning object traceback when raise as error with -W error -> print ResourceWarning object traceback when raised as error with -W error and -X tracemalloc
2019-06-20 16:16:27graingertcreate