Attached is a patch that adds a few features to the Python debugging support for gdb:

- listing of globals
- python breakpoints
- python stepping/stepping over/finishing of frames/running/continuing
- python code execution in the most recent Python frame (relative to the selected frame)

Unit tests are included in Lib/test/test_gdb.py. It should be compatible with both python 3 and python 2 (this is important because libpython.py is also part of the Cython debugger branch, see https://github.com/markflorisson88/cython/blob/master/Cython/Debugger/libpython.py ). Python 2 tests are in that Cython branch.

It would be great if libpython.py could be installed as an actual Python  module instead of a tool, where 'python-gdb.py' would be the actual tool that imports libpython.py. This may remove the need in the future to duplicate files in the Python source distribution and in future versions of Cython. Another good thing about that is that if users don't have python-gdb.py installed properly, or would like to use functionality without having loaded the interpreter in gdb (i.e. 'file python'), they can just do 'python import libpython' in gdb to load the Python support.

I can't assign this issue, but Dave Malcolm (dmalcolm) asked me to assign the issue to him, so I kindly request someone with these capabilities to assign this issue accordingly.

I forgot to mention, this patch works with gdb 7.2 or higher, but it does not prevent using other libpython functionality with gdb 7.1 or running the tests with gdb 7.1.

This is very interesting work - thank you!

Sorry for not commenting earlier (very busy), so here are my thoughts so far.

The baseline for the diff appears to be against the py3k branch, in that it adds back in classes from 2.*: PyIntObject.

There's currently some deliberate divergence between the code in the 2.7 branch and the py3k branch.  For example, in the 2.7 branch, we pretty-print PyUnicodeObject instances with a u'' prefix, whereas in the py3k branch we pretty-print them plainly: the code is trying to mimic the underlying syntax of a particular version of the python language in the inferior process.  (In both cases though, the gdb code is for python 2)

Are you attempting to support both python 2 and python 3 (in the inferior process) from the same code?

Going through the individual parts of the patch:

Do the changes to "pretty_printer_lookup" slow things down, I wonder?  (It has to be called every time gdb print a pointer)

"py-globals" looks like a nice addition, though I feel there should be an abstract base class for PyLocals and PyGlobals to inherit from ("PyNamespaceCommand", perhaps, though that sucks as a name).

The "py-break" command looks very useful  Unfortunately, AIUI we're in feature freeze for 3.2 now (sorry not to get to this earlier), and I have some fears about compatibility with all of the different versions of gdb 7 that are out there.  What versions of gdb have you tested this with?

_LoggingState looks interesting; getting the various versions of gdb to capture output as a python string can be challenging.  In particular, I've run into problems with buffering of large amounts of output; see: https://bugzilla.redhat.com/show_bug.cgi?id=620930

get_selected_inferior() clearly is impacted by gdb bugs, so I'm warying about merging that
Likewise for GenericCodeStepper.init_breakpoints (probably should refrain from swearing in comments, if nothing else, it's a dead give away that the code needs more work :)  I don't know if we have any rules about it, but I don't remember seeing such "colorful" language before in python's sources)

I notice that the selftests use skip_unless_7_2_decorator on the tests for py-break, py-step, py-next and py-exec.  If these commands only work on a sufficiently recent version of gdb, should the commands themselves also be wrapped in a conditional?   Either not registering them if gdb doesn't have the support, or perhaps registering a fallback that says "gdb not recent enough", or somesuch.  Part of gdb's UI is tab-completion, and it's nice for "py-[TAB]" to present all useful options.  That makes me think that we shouldn't register commands that are known to be incompatible with the running gdb version.

This is exciting work, in that gdb seems to be getting significantly more powerful, and the debug hooks are advancing to take advantage of it.  Unfortunately, given that Tools/gdb/libpython.py is part of the python source tree, it's arguably governed by the same feature freeze rules as the rest of Python (e.g. no new features in 2.7).  I wonder if an exception can be made for it, given that this code is a support tool that runs in a different process to the main python build, potentially for a different major-release of python.   (At some point we'll want to port gdb to python 3, which I'm not looking forward to...)

Indeed, I'm trying to make the code Python 2 and Python 3 (for the inferior) compatible, it's not really hard but indeed, the 'u' (Python 2) and 'b' (Python 3) stuff need special casing. Python 2 compatibility was also the reason why the PyIntObjectPtr class was merged back. I will make a patch that's compatible with both Python 2 and 3 (and without any "colorful" code :)) as this would be most preferably I think (even if it's shipped with Python 3, users might still want to use it for Python 2, and it's also easier for the Cython debugger which wants to be compatible with 2.5+).

As for the gdb version, I have tested with 7.1 (in which case the introduced commands won't work as they use stuff from the 7.2 API, in which case test_gdb.py also skips those tests) and 7.2. I agree that the functionality should be left out if it cannot work properly.

Indeed, the _LoggingState redirects logging to a file and then reads the output after the command returns, I've been using it successfully with big amounts of output and I don't think there should be a problem as redirection and pagination should be unrelated. The good thing about _LoggingState is that it actually captures *all* output (and it's fully reentrant), which the 'to_string' argument was not taking care of properly in 7.2 (it's fixed in the archer branch).

Indeed, gdb.get_selected_thread() is entirely broken in 7.2 (again, fixed in archer) but as you can see, it's not used (gdb.inferiors()[0] "works").

I'm currently looking into making stepping over faster and by extension stepping which we discussed earlier (I also discussed it with Tom Tromey previously). As you know, currently stepping over (and stepping) works with a "step-over loop" which might be turned into a "set a conditional breakpoint or watchpoint + continue" solution, which would mean a lot less context switches. I've not looked too serious into this matter, but I'll hope to get around to that soonish and I'll provide a new patch with all the corrections and improvements.
Another issue I'm fixing is the determination of the type of an arbitrary Python object, which was previously done with the Py_TPFLAGS_INT_SUBCLASS flags and friends. This is because they are new in 2.6 and I'd prefer to be 2.5 compatible (again, because I'm trying to keep the Cython debugger 2.5 compatible).

As for porting the gdb API to Python 3, I'm quite convinced that the API can be written in Cython, in which case it would mostly be a change in the build process rather than a serious code-refactoring issue. But I'll get around to that later...

Anyway, should I diff against the original libpython or against the original libpython + my previous diff?

Ok I attached a new patch that solves the things you mentioned. It can debug Python inferiors with versions 2.6+. Execution control commands (py-{run, cont, finish, step, next}) and py-exec need gdb 7.2+, py-break works with 7.1+.

It now also "supports exceptions", which means that if there is a pending exception when control is transferred to the debugger, the exception is printed (safely).
Also stepping and stepping over should be a lot faster as it now uses hardware watchpoints that watch the instruction pointer (f->f_lasti).

I forgot to remove a tempfile and reverted the Cython note at the top. A diff is provided (that should be applied upon the previously submitted patch). It's a diff because I don't have commit rights and svn does not support local commits.

Comments in msg123606 seem encouraging so I'm guessing this has just slipped under the radar.