Section 3.2 of 'The Python Language Reference' states:
  f_trace, if not None, is a function called at the start of each source code line

Run the attached tracer.py and see that although f_trace is None in both cases when 'cmd' is either
'delete' or 'set', the second case raises a TypeError exception:
$ python tracer.py
f_trace: None
delete start
delete done
f_trace: None
set start
Traceback (most recent call last):
  File "tracer.py", line 19, in <module>
    foo('set')
  File "tracer.py", line 15, in foo
    print(cmd, 'done')
  File "tracer.py", line 15, in foo
    print(cmd, 'done')
TypeError: 'NoneType' object is not callable

Also, the frame.f_lineno may be wrong in a traceback when f_trace is set to None because
PyFrame_GetLineNumber() does not handle this case.

The attached patch fixes this issue.
The patch also fixes issue 11992 and issue 20040.
The patch also fixes the dispatch_call() method of Bdb in the bdb module when the frame is a
generator and the previous command is next, until or return.
The patch also provides a backward compatible solution to the performance enhancement described in
issue 16672.

Adding the corresponding tests.

I think the code would be simpler if convert the argument of frame_settrace() to NULL if it is Py_None.

On a 'call' trace event, the bdb dispatch_call() function returns None when there is nothing to trace in this function (no breakpoints). With the changes made by the patch in ceval.c, the costly maybe_call_line_trace() function is not called on each line in this case since f->f_trace is Py_None. This provides the performance enhancements described in issue 16672 without breaking the _hotshot extension module or other extension modules using PyEval_SetTrace().

I agree that the code would be much simpler when f->f_trace is set to NULL by the frame_settrace() setter or trace_trampoline() when it is Py_None. The performance gain described above by using Py_None may not be worth the complexity.

Thanks for looking into this Serhiy.

This may be worthwhile optimization, but this is different issue. Let first fix a TypeError, and then open new issue for the optimization.

This patch fixes the TypeError.
New issue 27218: improve tracing performance with f_trace set to Py_None.

LGTM.

New changeset 4d916be61d46 by Serhiy Storchaka in branch '2.7':
Issue #20041: Fixed TypeError when frame.f_trace is set to None.
https://hg.python.org/cpython/rev/4d916be61d46

New changeset 74ad78d2dd8d by Serhiy Storchaka in branch '3.5':
Issue #20041: Fixed TypeError when frame.f_trace is set to None.
https://hg.python.org/cpython/rev/74ad78d2dd8d

New changeset f993dbeb2ad2 by Serhiy Storchaka in branch 'default':
Issue #20041: Fixed TypeError when frame.f_trace is set to None.
https://hg.python.org/cpython/rev/f993dbeb2ad2