It seems that using f_trace in the f_lineno getter PyFrame_GetLineNumber(), as
the condition to decide when tracing is active, is incorrect.
See the following two examples.

In the backtrace printed by tracer.py running with python 3.3,
the last entry should be line 12 instead of line 10:

$ python3 /tmp/tracer.py
Traceback (most recent call last):
  File "/tmp/tracer.py", line 15, in <module>
    foo()
  File "/tmp/tracer.py", line 10, in foo
    bar()
ZeroDivisionError: division by zero


This simple case does not occur with pdb, because pdb takes care of deleting
the f_trace attribute of all the frames in the call stack when removing the
trace function (see set_continue() in bdb.py). But this is not good enough when
a generator is involved as can be seen when running generator.py. In the
backtrace the last entry should be line 6 instead of line 8:

$ python3 /tmp/generator.py
> /tmp/generator.py(16)<module>()
-> foo()
(Pdb) step
--Call--
> /tmp/generator.py(10)foo()
-> def foo():
(Pdb) step
> /tmp/generator.py(11)foo()
-> it = gen()
(Pdb) step
> /tmp/generator.py(12)foo()
-> next(it)
(Pdb) step
--Call--
> /tmp/generator.py(3)gen()
-> def gen():
(Pdb) return
--Return--
> /tmp/generator.py(8)gen()->0
-> yield i
(Pdb) step
> /tmp/generator.py(13)foo()
-> next(it)
(Pdb) continue
Traceback (most recent call last):
  File "/tmp/generator.py", line 16, in <module>
    foo()
  File "/tmp/generator.py", line 13, in foo
    next(it)
  File "/tmp/generator.py", line 8, in gen
    yield i
ZeroDivisionError: division by zero


It seems that it could be possible to fix this issue by replacing the test for
f->f_trace in PyFrame_GetLineNumber, by a test for f->f_tstate->use_tracing,
and updating accordingly the f_lineno and f_trace setters.

Xavier, could you possibly provide a patch and a test?

The patch (on the default branch) reverts one of the changes made in r72488 to
introduce the new PyFrame_GetLineNumber() function (issue 5954): tb_lineno is
now back again the result of the call to PyCode_Addr2Line() instead of the
call to PyFrame_GetLineNumber().

The other changes made by r72488 in _warnings.c and ceval.c should also
probably be reverted as well.

The patch updates bdb set_continue() for consistency.

The patch adds a test to test_sys_settrace.

The proposed patch fixes the backtrace line numbers issue, but it does not fix
PyFrame_GetLineNumber() which is the recommended way to get the frame line
number.

As mentionned in the original message, testing for f->f_trace to implement the
f_lineno getter is not correct. The f_lineno setter is also wrong in allowing to
modify f_lineno when the frame is not the one that is being traced (pdb prevents
that to happen though, in do_jump()).

I am working on another patch that should fix the issue by changing
PyFrame_GetLineNumber() and the f_lineno accessors.

> fix the issue by changing PyFrame_GetLineNumber() and the f_lineno accessors

The new patch named traced_frame.patch has been uploaded.

Also, now it is not allowed anymore to set the f_lineno attribute of a frame
that is not the frame being traced, as f_lasti is invalidated anyway on
returning to the evaluation of that frame.

The traced_frame.patch fixes also issue 7238 and issue 16482.

The previous patch changed a field in the PyThreadState structure. This new patch is simpler and does not prevent to change f_lineno when it is not the attribute of the frame being traced. The new patch fixes also issue 7238, issue 16482 and issue 17697.

The patch is wrong, the frame may not be run by the current PyThreadState.