classification
Title: Systemtap and DTrace support
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: lukasz.langa Nosy List: SilentGhost, benesch, berker.peksag, bkabrda, cburroughs, cstratak, dmalcolm, durin42, jcea, koobs, lukasz.langa, python-dev, xmorel
Priority: normal Keywords: patch

Created on 2014-05-27 13:06 by bkabrda, last changed 2016-11-21 11:48 by cstratak. This issue is now closed.

Files
File name Uploaded Description Edit
21590_2.diff SilentGhost, 2016-10-29 12:43 review
create-Include-dir-to-properly-generate-pydtrace_probes.h-file.patch cstratak, 2016-11-08 16:43 review
Messages (14)
msg219219 - (view) Author: Bohuslav "Slavek" Kabrda (bkabrda) * Date: 2014-05-27 13:06
This is a tracking bug for development of combined systemtap and dtrace patch for Python. The separate patches were submitted at [1] (systemtap) and [2] (dtrace).
Since it was agreed that it'd be best to merge the two patches to reuse as much code as possible between them, I'm opening this issue to track the progress and discuss possible improvements and suggestions.

[1] http://bugs.python.org/issue14776
[2] http://bugs.python.org/issue13405
msg219222 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2014-05-27 13:18
Bohuslav, do you have Jabber/XMPP?. Mine is "jcea@jabber.org". This thing will require real time communication.

I rather prefer XMPP/Jabber, but I could accept IRC :-)
msg219342 - (view) Author: Bohuslav "Slavek" Kabrda (bkabrda) * Date: 2014-05-29 08:48
I'd really prefer IRC, if at all possible for you :)
When online, I'm usually on #fedora-python, but I can really come any channel of your choice.

Sorry for the delays in communication, I've got a lot on my plate right now and I'll probably not be very quick to respond
msg233126 - (view) Author: Nikhil Benesch (benesch) Date: 2014-12-27 06:57
Since it looks like development on this has stalled, I took the liberty of compiling all prior patches

    https://github.com/benesch/python-dtrace-tracker

and am actively working on merging them into one superpatch. I am neither a Python nor a DTrace expert, so I'm happy to step aside should one of you more experienced folks have something ready.

A few thoughts so far:

    * SPARC support? Can we drop this? It seems to add a bunch of intrusive
      inlined NOPs—that I'm not sure are even necessary.

    * @jcea, is this the original DTrace patch? https://github.com/benesch/python-dtrace-tracker/blob/master/dtrace-solaris-oracle-24.patch

      I'm trying to determine where those NOPs came from.

Cheers!
msg275543 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2016-09-10 02:22
Landed during the 3.6 beta1 sprint:
https://hg.python.org/cpython/rev/d622dbd71f2b

Thanks all for the preliminary tasks! I rebuilt the patch from first principles, reusing large parts of Jesús' work, as well as work by Dave and Nikhil.

Things deliberately left out for simplicity:
- ustack helpers, I have no way of testing them at this point since
they are Solaris-specific, don't work on macOS or Linux
- PyFrameObject * in function__entry/function__return, this is
SystemTap-specific and won't compile with DTrace
- SPARC support
- dynamic tracing
- sys module dtrace facility introspection (as described in the documentation, there's facilities in both Linux and macOS to do the same thing)

All of those might be added later.

I also left out instance new/delete probes for now since I had problems making them stable. We might still be able to squeeze them in if we're quick about it.

Dave, if you have an idea how to integrate your FrameObject pointer in a way that compiles for DTrace, too, this is the time to speak up :)

Jesús, if you'd like to add the ustack helper back in a separate patch and add to the documentation how this can be tested, that's great, too. I just didn't feel comfortable committing code myself that I have no way of testing, especially that it required more code (the offsets, including the generated .h here and there, etc.).
msg275623 - (view) Author: Kubilay Kocak (koobs) Date: 2016-09-10 11:12
@Łukasz in case you're not aware, all koobs-freebsd* bots are DTrace enabled, and can be tested with the custom builder. I'm on IRC (python-dev) if you need anything from me to help progress this
msg279670 - (view) Author: SilentGhost (SilentGhost) * Date: 2016-10-29 10:35
Building the documentation on Ubuntu 16.10, I started getting the following warnings:

/Doc/howto/instrumentation.rst:58: WARNING: Could not lex literal_block as "python3". Highlighting skipped.
/Doc/howto/instrumentation.rst:139: WARNING: Could not lex literal_block as "c". Highlighting skipped.
/Doc/howto/instrumentation.rst:211: WARNING: Could not lex literal_block as "c". Highlighting skipped.
/Doc/howto/instrumentation.rst:328: WARNING: Could not lex literal_block as "c". Highlighting skipped.

I guess the trick of using c for systemtap doesn't work anymore, not entirely sure what's the problem with the python3 highlighting.
msg279672 - (view) Author: SilentGhost (SilentGhost) * Date: 2016-10-29 12:43
Here is the patch that fixes warning in /Doc/howto/instrumentation.rst as well as cleans up a few nits here and there. I've removed attempt at highlighting using c since that doesn't work in practice.
msg279932 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2016-11-02 16:43
Fedora so far has been using the systemtap patch downstream from dmalcolm [0].

So for 3.6 by removing the systemtap patch (it cannot be applied anymore cleanly) and by enabling the --with-dtrace configure flag I get this error [1]:
<snip>
make: *** [Makefile:884: Include/pydtrace_probes.h] Error 1
<snip>

If however I touch the pydtrace_probes.h file I get various undefined references [2] and also:
<snip>
make: *** [Makefile:724: Programs/_freeze_importlib] Error 1
<snip>

[0] http://pkgs.fedoraproject.org/cgit/rpms/python3.git/tree/00055-systemtap.patch
[1] https://kojipkgs.fedoraproject.org//work/tasks/9656/16279656/build.log
[2] https://kojipkgs.fedoraproject.org//work/tasks/9670/16279670/build.log
msg280162 - (view) Author: Roundup Robot (python-dev) Date: 2016-11-06 18:30
New changeset d05d312161f2 by Berker Peksag in branch '3.6':
Issue #21590: Silence Sphinx warnings in instrumentation.rst
https://hg.python.org/cpython/rev/d05d312161f2

New changeset 442453fa3370 by Berker Peksag in branch 'default':
Issue #21590: Merge from 3.6
https://hg.python.org/cpython/rev/442453fa3370
msg280163 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-11-06 18:32
Thanks. SilentGhost. I went with ".. highlight:: shell-session" to simplify the patch a bit.

Łukasz, can we close this one and create new issues for further improvements now?
msg280254 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2016-11-07 22:33
Yes, let's close this.

@Charalampos, the error in your build comes from the /usr/bin/dtrace wrapper:
Traceback (most recent call last):
  File "/usr/bin/dtrace", line 440, in <module>
    sys.exit(main())
  File "/usr/bin/dtrace", line 385, in main
    providers.probe_write(s_filename, filename + suffix)
  File "/usr/bin/dtrace", line 181, in probe_write
    hdr = open(header, mode='w')
FileNotFoundError: [Errno 2] No such file or directory: 'Include/pydtrace_probes.h'

For some reason the path Include/pydtrace_probes.h is not writable. Maybe this is RPM-specific?
msg280318 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2016-11-08 16:43
@Łukasz

Dug a bit more to it.

Yes it is RPM specific for that case, in the sense that we create different subfolders for the debug and the normal(optimized) builds under the build/ dir, where the Include directory does not exist. the dtrace wrapper assumes that the directory already exists. The issue of course can be reproduced with any packaging system (or ways to compile python) that follow this approach, although I am not aware currently if this can happen or might be happening anywhere else.

By creating the directory (if it doesn't exist) at the Makefile rule for generating pydtrace_probes.h, this can be circumvented.

Attaching a patch that fixes this issue for consideration.
msg281350 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2016-11-21 11:48
@Łukasz

Would it be possible to review the patch?

Or is it preferable to open a new issue?
History
Date User Action Args
2016-11-21 11:48:12cstrataksetmessages: + msg281350
2016-11-08 16:43:58cstrataksetfiles: + create-Include-dir-to-properly-generate-pydtrace_probes.h-file.patch

messages: + msg280318
2016-11-08 06:43:27berker.peksagsetstage: patch review -> resolved
2016-11-07 22:33:56lukasz.langasetstatus: open -> closed
resolution: fixed
2016-11-07 22:33:49lukasz.langasetmessages: + msg280254
2016-11-06 18:32:25berker.peksagsetmessages: + msg280163
2016-11-06 18:30:27python-devsetnosy: + python-dev
messages: + msg280162
2016-11-02 16:43:29cstrataksetnosy: + cstratak
messages: + msg279932
2016-10-29 12:43:06SilentGhostsetfiles: + 21590_2.diff

versions: + Python 3.7
keywords: + patch
nosy: + berker.peksag

messages: + msg279672
stage: patch review
2016-10-29 10:35:25SilentGhostsetnosy: + SilentGhost
messages: + msg279670
2016-09-10 11:12:32koobssetmessages: + msg275623
2016-09-10 11:09:03koobssetnosy: + koobs
2016-09-10 02:22:18lukasz.langasetversions: + Python 3.6, - Python 3.5
nosy: + lukasz.langa

messages: + msg275543

assignee: lukasz.langa
2016-09-10 02:15:33lukasz.langalinkissue14776 superseder
2016-09-10 02:13:31lukasz.langalinkissue13405 superseder
2015-09-02 19:18:34cburroughssetnosy: + cburroughs
2015-01-14 15:00:43durin42setnosy: + durin42
2014-12-27 06:57:43beneschsetnosy: + benesch
messages: + msg233126
2014-06-26 08:51:48xmorelsetnosy: + xmorel
2014-05-29 08:48:59bkabrdasetmessages: + msg219342
2014-05-27 13:43:16dmalcolmsetnosy: + dmalcolm
2014-05-27 13:18:29jceasetmessages: + msg219222
2014-05-27 13:06:35bkabrdacreate