classification
Title: Add Py_BREAKPOINT and sys._breakpoint hooks
Type: enhancement Stage: patch review
Components: Interpreter Core, Library (Lib) Versions: Python 3.2, Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: cheryl.sabella, dmalcolm, loewis, pitrou, vstinner
Priority: normal Keywords: patch

Created on 2010-08-18 23:43 by dmalcolm, last changed 2018-02-27 20:48 by dmalcolm.

Files
File name Uploaded Description Edit
add-sys.breakpoint.patch dmalcolm, 2010-08-18 23:43 Patch against py3k branch review
py3k-add-breakpoint-2010-11-01-001.patch dmalcolm, 2010-11-01 21:14 review
Messages (10)
msg114301 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-08-18 23:43
It's sometimes useful to be able to programatically inject a breakpoint when debugging CPython.

For example, sometimes you want a conditional breakpoint, but the logic involved is too complex to be expressed in the debugger (e.g. runtime complexity of evaluating the conditional in the debugger process, or deficiency of the debugger itself).

I'm attaching a patch which:
  - adds a Py_BREAKPOINT macro to pyport.h   This is available as a quick and dirty way of hardcoding a breakpoint in code (e.g. in extension modules); so that when you need to you can put of these in (perhaps guarded by C-level conditionals):
       if (complex_conditional()) {
           Py_BREAKPOINT();
       }

  - when Py_BREAKPOINT is defined, adds a sys.breakpoint() method.  This means that you can add C-level breakpoints to Python scripts, perhaps guarded by python-level conditionals:
       if foo and bar and not baz:
          sys.breakpoint()

Naturally this is highly system-dependent.   Only tested on Linux (Fedora 13 x86_64 with gcc-4.4.4).  The Windows implementation was copied from http://bugs.python.org/issue8863 but I don't have a Windows box to test it on.

I note that the GLib library within GNOME has a similar idea with a G_BREAKPOINT macro, which has accumulated a collection of platform-specific logic:
  http://git.gnome.org/browse/glib/tree/glib/gbacktrace.h
(unfortunately that's LGPLv2+ licensed)

Doesn't yet have a unit test.

Note that when running on Linux when _not_ under a debugger, the default for SIGTRAP is to get a coredump:
   Trace/breakpoint trap (core dumped)
so people should be strongly discouraged from adding these calls to their code.
msg114303 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-08-18 23:50
> Note that when running on Linux when _not_ under a debugger, the 
> default for SIGTRAP is to get a coredump:
>   Trace/breakpoint trap (core dumped)
> so people should be strongly discouraged from adding these calls to
> their code.
Looks like Windows' DebugBreak has similar behavior here; according to:
  http://msdn.microsoft.com/en-us/library/ms679297(VS.85).aspx
"If the process is not being debugged, the function uses the search logic of a standard exception handler. In most cases, this causes the calling process to terminate because of an unhandled breakpoint exception."
msg120174 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-11-01 21:14
Adding updated version of patch, which adds documentation to sys.rst and adds a unit test.

I'm a little wary of this: it seems useful but also too much like a self-destruct button for my taste.
msg120175 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2010-11-01 21:15
I renamed it from sys.breakpoint to sys._breakpoint, since this is CPython-specific
msg120241 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-11-02 17:25
I would rename Py_BREAKPOINT to _Py_BREAKPOINT since we don't really want to support this. Also, why do you allow any arguments to sys._breakpoint()?
msg167169 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2012-08-01 21:22
Note to self: a messy way of forcing gdb to do the equivalent of a breakpoint directly from Python is:
  os.kill(os.getpid(), signal.SIGTRAP)
msg167623 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2012-08-07 16:17
On Tue, 2010-11-02 at 17:25 +0000, Antoine Pitrou wrote:
> Antoine Pitrou <pitrou@free.fr> added the comment:
> 
> I would rename Py_BREAKPOINT to _Py_BREAKPOINT since we don't really want to support this. Also, why do you allow any arguments to sys._breakpoint()?
Agreed about _Py_BREAKPOINT.

The reason for allowing arguments to sys._breakpoint() is so that the
developer can pass in arbitrary objects (or collections of objects),
which can then be easily inspected from the debugger.  Does that seem
sane?

Maybe the docs should read:

------
This may be of use when tracking down bugs: the breakpoint can be
guarded by Python-level conditionals, and supply Python-generated data::

   if foo and bar and not baz:
       sys._breakpoint(some_func(foo, bar, baz))

In the above example, if the given python conditional holds (and no
exception is raised calling "some_func"), execution will halt under
the debugger within Python/sysmodule.c:sys_breakpoint, and the result of
some_func() will be inspectable in the debugger as
((PyTupleObject*)args)[0]

   static PyObject *
   sys_breakpoint(PyObject *self, PyObject *args)
   {
     _Py_BREAKPOINT();
     Py_RETURN_NONE;
   }

It can also be useful to call when debugging the CPython interpreter: if
you add a call to this function immediately before the code of interest,
you can step out of sys_breakpoint and then step through subsequent
execution.
------

I thought about it making it METH_O instead (to make it easier to look
at a single object), but then you'd be forced to pass an object in when
using it, I think (though None should work).
msg167751 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-08-09 01:22
> I thought about it making it METH_O instead (to make it easier to
> look at a single object), but then you'd be forced to pass an object
> in when using it, I think (though None should work).

I don't like this API. I really prefer METH_O because it is already very difficult to dump a Python object in a debugger (especially with gdb macros). Why not using two functions? One without argument, One with an argument.

The sys module is maybe not the right place for such low level function. You may create a new module with a name starting with _ (ex: _python_dbg). I'm quite sure that you will find other nice functions to add if you have a module dedicated to debugging :-)

If you don't want to create a new module, faulthandler might be used to add new debug functions.

It's maybe unrelated, but Brian Curtin also created a module to help debugging with Visual Studio:
https://bitbucket.org/briancurtin/minidumper/

We may use the same module for all debugging functions? (I suppose that Brian Curtin wants to integrate its minidumper into Python 3.4.)

Example:

 * faulthandler._breakpoint()
 * faulthandler._inspect(obj) # breakpoint with an object
 * faulthandler.enable_minidumper(...)
 * faulthandler.disable_minidumper(...)

--

For Power PC, the machine code to generate a breakpoint is 0x0cc00000. The instruction looks to be called twllei. Something like: "twllei reg_ZERO,trap_Cerror". Hum, it looks like there is family of instructions related to trap: all TW* instructions (twle, twlt, twge, ...).
msg312605 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2018-02-23 00:16
Did PEP553 make this issue obsolete?
msg313024 - (view) Author: Dave Malcolm (dmalcolm) (Python committer) Date: 2018-02-27 20:48
On Fri, 2018-02-23 at 00:16 +0000, Cheryl Sabella wrote:
> Cheryl Sabella <chekat2@gmail.com> added the comment:
> 
> Did PEP553 make this issue obsolete?

I *think* they have slightly different scope: if I'm reading it right,
PEP553 is about injecting a breakpoint into the Python debugger.  This
proposal was about injecting a lower-level breakpoint for debugging
CPython itself (for e.g. gdb to handle).

The idea was to make it easier to, say, step through a particular
CPython construct at the C level by injecting a breakpoint right before
it:

def test_something():
  # lots of setup
  sys.c_level_breakpoint()
  # whatever comes next

so that sys.c_level_breakpoint drops you into, say, gdb, and from there
you can step through the following Python code at the C level, without
having to express stepping through all the setup at the C/gdb level.

Hope that makes sense.

That said, I'm full-time on gcc these days, and unlikely to pursue this
from the CPython side.
History
Date User Action Args
2018-02-27 20:48:25dmalcolmsetmessages: + msg313024
2018-02-23 00:16:09cheryl.sabellasetnosy: + cheryl.sabella
messages: + msg312605
2012-08-09 01:22:05vstinnersetmessages: + msg167751
2012-08-07 16:17:27dmalcolmsetmessages: + msg167623
2012-08-01 21:22:50dmalcolmsetmessages: + msg167169
2011-03-15 18:40:16dmalcolmsetnosy: + loewis
2010-11-02 17:25:37pitrousetnosy: + pitrou
messages: + msg120241
2010-11-01 21:15:02dmalcolmsetmessages: + msg120175
title: Add Py_BREAKPOINT and sys.breakpoint hooks -> Add Py_BREAKPOINT and sys._breakpoint hooks
2010-11-01 21:14:06dmalcolmsetfiles: + py3k-add-breakpoint-2010-11-01-001.patch

messages: + msg120174
2010-08-19 11:46:42eric.araujosetkeywords: patch, patch
title: RFE(patch): add Py_BREAKPOINT and sys.breakpoint hooks -> Add Py_BREAKPOINT and sys.breakpoint hooks
2010-08-18 23:50:54dmalcolmsetkeywords: patch, patch

messages: + msg114303
2010-08-18 23:43:23dmalcolmcreate