classification
Title: pdb continue followed by an exception in the same frame shows incorrect frame linenumber
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Romuald, Sriram Rajagopalan, bglsriram, georg.brandl, orsenthil, xdegaye
Priority: normal Keywords: patch

Created on 2016-04-16 13:08 by Sriram Rajagopalan, last changed 2021-04-19 13:32 by Romuald.

Files
File name Uploaded Description Edit
bdbfix.patch bglsriram, 2016-04-18 19:00 Fixes the issue with bdb set_continue
Messages (6)
msg263553 - (view) Author: Sriram Rajagopalan (Sriram Rajagopalan) Date: 2016-04-16 13:08
Consider this simple python program - 

  1 #!/usr/bin/python
  2
  3 import pdb
  4 import sys
  5 import traceback
  6
  7 def trace_exception( type, value, tb ):
  8     traceback.print_tb( tb )
  9     pdb.post_mortem( tb )
 10
 11 sys.excepthook = trace_exception
 12
 13 def func():
 14     print ( "Am here in func" )
 15     pdb.set_trace()
 16     print ( "Am here after pdb" )
 17     print ( "Am going to assert now" )
 18     assert False
 19     print ( "Am here after exception" )
 20
 21 def main():
 22     func()
 23
 24 if __name__ == "__main__":
 25     main()


On running this program - 

% ./python /tmp/test.py
Am here in func
> /tmp/test.py(16)func()
-> print ( "Am here after pdb" )
(Pdb) c
Am here after pdb
Am going to assert now
  File "/tmp/test.py", line 25, in <module>
    main()
  File "/tmp/test.py", line 22, in main
    func()
  File "/tmp/test.py", line 16, in func
    print ( "Am here after pdb" )
> /tmp/test.py(16)func()
-> print ( "Am here after pdb" ) >>>> This should have been at the line corresponding to "Am going to assert now"
(Pdb)


This seems to be an bug ( due to a performance consideration ) with the way python bdb's set_continue() has been implemented -

https://hg.python.org/cpython/file/2.7/Lib/bdb.py#l227

    def set_continue(self):
        # Don't stop except at breakpoints or when finished
        self._set_stopinfo(self.botframe, None, -1)
        if not self.breaks:
            # no breakpoints; run without debugger overhead
            sys.settrace(None)
            frame = sys._getframe().f_back
            while frame and frame is not self.botframe:
                del frame.f_trace
                frame = frame.f_back

Basically what happens after "c" in a "(Pdb)" prompt is that bdb optimizes for the case where there are no more break points found by cleaning up the trace callback from all the frames.

However, all of this happens in the context of tracing itself and hence the trace_dispatch function in https://hg.python.org/cpython/file/2.7/Lib/bdb.py#l45 still returns back the trace_dispatch as the new system trace function. For more details on sys.settrace(), check https://docs.python.org/2/library/sys.html#sys.settrace

Check, the function trace_trampoline at https://hg.python.org/cpython/file/2.7/Python/sysmodule.c#l353
which sets f->f_trace back to result at https://hg.python.org/cpython/file/2.7/Python/sysmodule.c#l377

This seems to be an bug ( due to a performance consideration ) with the way python bdb's set_continue() has been implemented -

https://hg.python.org/cpython/file/2.7/Lib/bdb.py#l227

    def set_continue(self):
        # Don't stop except at breakpoints or when finished
        self._set_stopinfo(self.botframe, None, -1)
        if not self.breaks:
            # no breakpoints; run without debugger overhead
            sys.settrace(None)
            frame = sys._getframe().f_back
            while frame and frame is not self.botframe:
                del frame.f_trace
                frame = frame.f_back

Basically what happens after "c" in a "(Pdb)" prompt is that bdb optimizes for the case where there are no more break points found by cleaning up the trace callback from all the frames.

However, all of this happens in the context of tracing itself and hence the trace_dispatch function in https://hg.python.org/cpython/file/2.7/Lib/bdb.py#l45 still returns back the trace_dispatch as the new system trace function. For more details on sys.settrace(), check https://docs.python.org/2/library/sys.html#sys.settrace

Check, the function trace_trampoline at https://hg.python.org/cpython/file/2.7/Python/sysmodule.c#l353
which sets f->f_trace back to result at https://hg.python.org/cpython/file/2.7/Python/sysmodule.c#l377

Now, check the function PyFrame_GetLineNumber() which is used by the traceback to get the frame line number https://hg.python.org/cpython/file/2.7/Objects/frameobject.c#l63

int
PyFrame_GetLineNumber(PyFrameObject *f)
{
    if (f->f_trace)
        return f->f_lineno;
    else
        return PyCode_Addr2Line(f->f_code, f->f_lasti);
}

Basically this function returns back the stored f->f_lineno in case the f->f_trace is enabled.

The fix is fortunately simple - 

Just set self.trace_dispatch to None if pdb set_continue decides to run without debugger overhead.
msg265833 - (view) Author: Sriram Rajagopalan (bglsriram) * Date: 2016-05-18 17:31
ping
msg266170 - (view) Author: Sriram Rajagopalan (bglsriram) * Date: 2016-05-23 17:55
ping.

Gentle reminder to review the patch
msg267239 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-06-04 06:56
See also issue 24565.
msg391377 - (view) Author: Romuald Brunet (Romuald) * Date: 2021-04-19 13:31
Ran into a similar issue today

This issue (and mine is fixed) in python 3.10. Most probaly related to #24565 fix
msg391378 - (view) Author: Romuald Brunet (Romuald) * Date: 2021-04-19 13:32
I meant: this issue is fixed in 3.10
History
Date User Action Args
2021-04-19 13:32:12Romualdsetmessages: + msg391378
2021-04-19 13:31:39Romualdsetnosy: + Romuald

messages: + msg391377
versions: + Python 3.7, Python 3.8, Python 3.9
2016-06-04 06:56:50xdegayesetnosy: + xdegaye
messages: + msg267239
2016-05-23 17:55:39bglsriramsetmessages: + msg266170
2016-05-18 17:31:06bglsriramsetnosy: + bglsriram
messages: + msg265833
2016-04-18 19:01:38Sriram Rajagopalansetfiles: - bdbfix.patch
2016-04-18 19:00:28bglsriramsetfiles: + bdbfix.patch
2016-04-16 13:21:01orsenthilsetnosy: + orsenthil
2016-04-16 13:18:40SilentGhostsetnosy: + georg.brandl

versions: - Python 3.2, Python 3.3, Python 3.4
2016-04-16 13:08:12Sriram Rajagopalancreate