It would be great if the main Python distribution supported DTrace on
(Open)Solaris / FreeBSD / OS X.

I've attached a patch against 2.6 that instruments function calls. It's
from the ed scripts included in Apple's Python distribution:
http://developer.apple.com/opensource/index.html (PSF License)

It is imperfect because I wasn't sure how to do the equivalent of IFDEF
in the Makefile.pre.in, as you can see this patch will make ceval.o
depend on pydtrace.h, which depends on having a local dtrace command.
The rest of the patch properly uses IFDEFs.

At one point we were told Apple would try to backport their dtrace
instrumentation. I don't know what the status of that is, but it
obviously has not happened.

It appears that Apple has dtracified their Python exeutable in Leopard.
Any chance that they can be persuaded to release a patch? I'm working
on a patch (based on some work from a friend at work), but it seems like
Apple already has a horse in the race. It would be nice if we could
see what they have done.

They have released the changes, that's what my patch (attached to the
issue) is based on.

It's working, it just needs to be cleaned up (it will fail, I believe,
for people on systems without DTrace - as I said I'm not very familiar
with Makefiles).

You can use it now against 2.6 and probably with very few changes
against 2.5.

Brett> They have released the changes, that's what my patch (attached to
Brett> the issue) is based on.

I see the reference to Apple in your original post, but can't find anything
related to dtrace & python starting from the URL you gave. Do you have
something more specific?

At this point Jeff's code does a fair bit more than simply tracing the
CALL_FUNCTION opcode but needs some work with the other CALL_FUNCTION_*
opodes. It also has some obmalloc tracing which I've not yet tested. I
would have thought Apple would more heavily instrument the interpreter than
it appears they have.

Brett> It's working, it just needs to be cleaned up (it will fail, I
Brett> believe, for people on systems without DTrace - as I said I'm not
Brett> very familiar with Makefiles).

I took care of the configure.in/Makefile.pre.in stuff today. The intent is
that you would configure using --enable-dtrace then not have to do anything
else to build a dtrace-aware interpreter.

Brett> You can use it now against 2.6 and probably with very few changes
Brett> against 2.5.  

We are using 2.4 at work and it works there. I'm fairly certain we should
be able to get it working for the entire 2.x and 3.x series with only a
little effort.

(I still need to get approval to release it, but I don't think that will be
a big deal. I've already alerted my boss and he's generally receptive to
such things.)

Skip

On Wed, Nov 12, 2008 at 9:31 PM, Skip Montanaro <report@bugs.python.org> wrote:

I see the reference to Apple in your original post, but can't find anything
related to dtrace & python starting from the URL you gave. Do you have
something more specific?

http://www.opensource.apple.com/darwinsource/10.5.5/python-30.1.2/

That isn't Python 3.0, that's what I guess they call Python "Apple
Version 30"? Anyways, it's their Python 2.5.1 and their "patches"
against it are actually ed scripts, contained in fix/

That's where my patch comes from, it's just changing the .ed scripts
to a patch and applying it against 2.6

At this point Jeff's code does a fair bit more than simply tracing the
CALL_FUNCTION opcode but needs some work with the other CALL_FUNCTION_*
opodes. It also has some obmalloc tracing which I've not yet tested. I
would have thought Apple would more heavily instrument the interpreter than
it appears they have.

Sun has released a patch against DTrace that probes more than just
function calls also. At least, it provides line number and some other
information, http://cvs.opensolaris.org/source/xref//jds/spec-files/trunk/patches/Python-07-dtrace.diff

I couldn't figure out why it broke the compile on OS X though, and
could only get it working on Solaris.

There's another problem (I think) with DTrace probes ... if the Apple
guys release Apple-Probe-Python.d and the Sun guys release
Sun-Probe-Python.d (just two scripts) and we setup our probes
differently, one or both will fail (because one may expect a probe for
foo while another expects a probe for bar). Kind of sucks,
considering Sun was first and Apple chose to probe Python differently.
Now we have to pick one or make a 3rd... I could be very mistaken
here, though.

Brett> http://www.opensource.apple.com/darwinsource/10.5.5/python-30.1.2/
...
Brett> http://cvs.opensolaris.org/source/xref//jds/spec-files/trunk/patches/Python-07-dtrace.diff

Thanks for the pointers. I'll work on getting a uniform patch which
incorporates both. (Though some of the comments in the OpenSolaris patch
are a bit scary.)

Brett> There's another problem (I think) with DTrace probes ... if the
Brett> Apple guys release Apple-Probe-Python.d and the Sun guys release
Brett> Sun-Probe-Python.d (just two scripts) and we setup our probes
Brett> differently, one or both will fail (because one may expect a
Brett> probe for foo while another expects a probe for bar).  Kind of
Brett> sucks, considering Sun was first and Apple chose to probe Python
Brett> differently.

Yeah, that would suck. It would be nice if they could agree on a common set
of probes. I don't know that we have any contacts within the two
development communities but if we can scare some up maybe we can get them to
talk to each other. :-/

Skip

On Thu, Nov 13, 2008 at 08:05, Skip Montanaro <report@bugs.python.org> wrote:

Skip Montanaro <skip@pobox.com> added the comment:

Brett> http://www.opensource.apple.com/darwinsource/10.5.5/python-30.1.2/
...
Brett> http://cvs.opensolaris.org/source/xref//jds/spec-files/trunk/patches/Python-07-dtrace.diff

Thanks for the pointers. I'll work on getting a uniform patch which
incorporates both. (Though some of the comments in the OpenSolaris patch
are a bit scary.)

Brett> There's another problem (I think) with DTrace probes ... if the
Brett> Apple guys release Apple-Probe-Python.d and the Sun guys release
Brett> Sun-Probe-Python.d (just two scripts) and we setup our probes
Brett> differently, one or both will fail (because one may expect a
Brett> probe for foo while another expects a probe for bar). Kind of
Brett> sucks, considering Sun was first and Apple chose to probe Python
Brett> differently.

Yeah, that would suck. It would be nice if they could agree on a common set
of probes. I don't know that we have any contacts within the two
development communities but if we can scare some up maybe we can get them to
talk to each other. :-/

Obviously Ronald is the Apple insider to talk to. If you want Sun, you
might want to talk to Ted Leung and see who he can put you in touch
with.

And courtesy of Philip Jenvey, here I am.

I would *really* like to work to help make this happen. Laszlo Peter
at Sun has been doing the ports of Python on Solaris, but we are not up
to 2.6 just yet. I'm attaching a pointer to his patches against 2.5:

http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/Python25-07-dtrace.diff

These patches include John Levon's ustack provider, which if you care
about DTrace and Python, you want to have.

I've also had some conversations with people at Apple, and they have
agreed that they will pull DTrace probes from python.org if they got in
there. That would solve the diverging probe problem.

I'd love to have a discussion about DTrace probe futures for CPython --
probably on python-dev.

I'm the python package maintainer at Sun.
We would really like to get the dtrace probes upstream, let me know how
I can help.

So I completely dropped the ball on this. It appears we have some
folks from Sun and Brett surmised that Ronald Oussoren would be the
likely person to do the heavy lifting on the Apple side of things.
Ronald, I've made you nosy. I will try to get the Solaris patch Ted
referenced running against the trunk (2.7) and py3k (3.1) code bases,
but it would be helpful if you could take a quick stab at it as well.

On Jan 22, 2009, at 1:35 PM, Skip Montanaro wrote:

Skip Montanaro <skip@pobox.com> added the comment:

So I completely dropped the ball on this. It appears we have some
folks from Sun and Brett surmised that Ronald Oussoren would be the
likely person to do the heavy lifting on the Apple side of things.
Ronald, I've made you nosy. I will try to get the Solaris patch Ted
referenced running against the trunk (2.7) and py3k (3.1) code bases,
but it would be helpful if you could take a quick stab at it as well.

Great. Actually the latest version of the patch is this one: <http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/Python26-07-dtrace.diff
>. I tried to quickly compile it on MacOS 10.5, Python 2.6.1, but
it didn't work right out of the box.

Ted

After applying the patch and reconfiguring I get compilation errors in
Python/ceval.c. I suspect it's because there is a new header file,
Python/python.h. That's probably found by the #include directive in
favor of Include/Python.h because of the Mac's case-insensitive file
system. Changing the Makefile targets and actions so it's named
python-dtrace.h seems to work a bit better. I get an error later
running dtrace which I have yet to investigate.

me> I get an error later running dtrace which I have yet to investigate.

Apple's dtrace program doesn't support the -G flag. When I remove it from
Makefile.pre.in and rebuild I get an error about privileges:

dtrace -o [Python/dtrace.o](https://github.com/python/cpython/blob/main/Python/dtrace.o)  -C -s ./Python/python.d [Python/ceval.o](https://github.com/python/cpython/blob/main/Python/ceval.o)
dtrace: failed to initialize dtrace: DTrace requires additional privileges

I tried adding "sudo" to the dtrace command. Then it prompts for my
password and emits this error:

sudo dtrace -o [Python/dtrace.o](https://github.com/python/cpython/blob/main/Python/dtrace.o)  -C -s ./Python/python.d [Python/ceval.o](https://github.com/python/cpython/blob/main/Python/ceval.o)
Password:
dtrace: failed to compile script ./Python/python.d: line 11: extraneous argument '[Python/ceval.o](https://github.com/python/cpython/blob/main/Python/ceval.o)' ($1 is not referenced)

I'm not sure what to do at this point. I'm not really a dtrace person.
Perhaps the Apple and Sun dtrace experts can offer a way out of this little
corner.

Skip

Please see here for discussion about the -G flag on OS X:
http://markmail.org/message/4nheqnexjr2o6mcx

If I read it correctly, on OS X, you will need to use -h
instead of -G and it won't emit an object file (dtrace.o)
so you will not need to link it.

Unfortunately, this means that the makefile will have to
be different on Solaris and OS X :(

Laca> Please see here for discussion about the -G flag on OS X:
Laca> http://markmail.org/message/4nheqnexjr2o6mcx

Laca> If I read it correctly, on OS X, you will need to use -h instead
Laca> of -G and it won't emit an object file (dtrace.o) so you will not
Laca> need to link it.

Laca> Unfortunately, this means that the makefile will have to be
Laca> different on Solaris and OS X :(

We can worm around that using configure. Thanks for the reference. That
will go a long way toward getting the Sun dtrace probes working on Mac OSX.
For reference by others, here's Lee Packham's dtrace patch for PostgreSQL on
Mac OSX: http://blog.leenux.org.uk/wp-content/uploads/2008/10/postgresql-825-fixosxdtracediff.gz

Here's a patch against the current trunk (2.7) which compiles on my Mac. It
adds a --with-dtrace configure option. The code checks to see if the -G
option is understood by the dtrace command. If so, dtrace support is added
Sun-style. If not we do things Apple's way.

The existing test cases pass on Apple except for one case in test_sys which
tries to confirm the size of a frame object. I added an #ifdef WITH_DTRACE
around the extra slot but there is no way of telling from Python code that
this extra slot is there. That would have to somehow be exposed to the
Python programmer so the test can be adjusted. All tests pass when
--with-dtrace is omitted. I have not yet tried this on Solaris. I will try
to get to it this week if someone doesn't beat me to it.

There are as yet no new dtrace test cases so I can't confirm that the added
dtrace support actually works. There are also no documentation updates.
Unlike most compile options this adds a significant new feature to the
runtime environment so some documentation changes are probably called for.

I tried building this on my Mac and got this;

ar cr libpython2.7.a Python/_warnings.o Python/Python-ast.o Python/
asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o Python/
compile.o Python/codecs.o Python/errors.o Python/frozen.o Python/
frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o
Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/
graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/
modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o
Python/pyarena.o Python/pyfpe.o Python/pymath.o Python/pystate.o
Python/pythonrun.o Python/structmember.o Python/symtable.o Python/
sysmodule.o Python/traceback.o Python/getopt.o Python/pystrcmp.o
Python/pystrtod.o Python/formatter_unicode.o Python/formatter_string.o
Python/dynload_shlib.o @DTRACEOBJS@ Python/mactoolboxglue.o Python/
thread.o
ar: @DTRACEOBJS@: No such file or directory
make: *** [libpython2.7.a] Error 1

my configure line was:

./configure --enable-framework --enable-toolbox-glue --with-threads --
enable-dtrace

On Jan 25, 2009, at 2:00 PM, Skip Montanaro wrote:

Skip Montanaro <skip@pobox.com> added the comment:

Here's a patch against the current trunk (2.7) which compiles on my
Mac. It
adds a --with-dtrace configure option. The code checks to see if
the -G
option is understood by the dtrace command. If so, dtrace support
is added
Sun-style. If not we do things Apple's way.

The existing test cases pass on Apple except for one case in
test_sys which
tries to confirm the size of a frame object. I added an #ifdef
WITH_DTRACE
around the extra slot but there is no way of telling from Python
code that
this extra slot is there. That would have to somehow be exposed to
the
Python programmer so the test can be adjusted. All tests pass when
--with-dtrace is omitted. I have not yet tried this on Solaris. I
will try
to get to it this week if someone doesn't beat me to it.

There are as yet no new dtrace test cases so I can't confirm that
the added
dtrace support actually works. There are also no documentation
updates.
Unlike most compile options this adds a significant new feature to the
runtime environment so some documentation changes are probably
called for.

Added file: http://bugs.python.org/file12861/dtrace.diff

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue4111\>

---

Ted> I tried building this on my Mac and got this;

Forgive me if I'm preaching to the choir here.

Did you run autoconf or autoreconf after applying the patch? If not,
@DTRACEOBJS@ would not be a substitutable string. It's fairly common (at
least in the Python community) to omit modified configure scripts from these
sorts of patches because the changes to generated configure scripts between
different versions of autoconf are so massive that they dwarf the actual
functional changes in the patch, often by a couple orders of magnitude.

Skip

I didn't run auto(re)conf. After I did that, all was well. However,
the ustack provider doesn't appear to be working correctly. I tried
running the py_profile.d from the DTrace toolkit, and it doesn't show
any stack traces, and when the script starts up it says

cc1: warning: /dev/fd/5 is shorter than expected

The basic function entry/exit probes appear to be working.

John +nosy'ed himself, so perhaps he'll have some insight?

On Jan 27, 2009, at 3:39 PM, Skip Montanaro wrote:

Skip Montanaro <skip@pobox.com> added the comment:

Ted> I tried building this on my Mac and got this;

Forgive me if I'm preaching to the choir here.

Did you run autoconf or autoreconf after applying the patch? If not,
@DTRACEOBJS@ would not be a substitutable string. It's fairly
common (at
least in the Python community) to omit modified configure scripts
from these
sorts of patches because the changes to generated configure scripts
between
different versions of autoconf are so massive that they dwarf the
actual
functional changes in the patch, often by a couple orders of
magnitude.

Skip

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue4111\>

---

I haven't seen "shorter than expected" message before, sounds like a Mac
OS X specific thing.

As for never getting any probes out, this is where it gets fun.
Debugging this is very tricky indeed: it involves you dropping into the
kernel debugger and looking at the interpreter's state. Most likely, the
compiler is doing something different with the relevant Python
functions, causing the logic in the DTrace ustack probe to fail.

I don't even know how the Apple guys debug this sort of thing, but
they'd definitely need to be involved.

Skip, it doesn't appear that the ustack helper is getting incorporated
into the OS X build anywhere. This rule is obviously wrong (compiling
the wrong input file with the wrong flags):

Include/phelper.h: $(srcdir)/Include/phelper.d
dtrace -o $@ $(DFLAGS) -C -h -s $(srcdir)/Python/python.d

I *think* it should be something along these lines:

Include/phelper.h: $(srcdir)/Include/phelper.d
dtrace -o $@ -DPHELPER $(DFLAGS) $(CPPFLAGS) -C -h -s
$(srcdir)/Include/phelper.d

There are some static functions in the standard system headers that get
pulled in that need to be avoided. The #define _SYS_STAT_H up at the top
avoids similar problems on Solaris. I'm working through the OS X headers
now, but I'm stuck on trying to avoid sys/_structs.h. Unfortunately, it
does not have the #ifndef _SYS__STRUCTS_H magic up at the top that would
allow me to do so, nor any convenient #ifdefs around the offending
declaration. <sigh>

Even if that would work, phelper.h is not #included anywhere. I'm not
really sure where it needs to go. On Solaris, the object file gets
linked in, but on OS X, you don't generate an object file; you just
#include the .h. Somewhere. I'm guessing where the pydtrace.h is
currently, but I don't really know. John, do you have any insights?

Also, s/DTRADEHDRS/DTRACEHDRS/

Got a bit farther. Adding this stanza to the top of phelper.d gets past
the issues in the headers:

#ifdef __APPLE__
#define _SYS_TIME_H_
#define _SYS_SELECT_H_
#define __MATH_H__
#define _OS__OSBYTEORDER_H
#define _FD_SET
#define __GNUC_VA_LIST
#endif /* __APPLE__ */

However, I now get a more legitimate dtrace compilation error that John
might be able to help us interpret:

$ dtrace -o /Users/rkern/hg/Python-2.5.4/Include/phelper.h -DPHELPER 
-I. -IInclude -I/Users/rkern/hg/Python-2.5.4/Include  -C -h -s
/Users/rkern/hg/Python-2.5.4/Include/phelper.d
dtrace: failed to compile script
/Users/rkern/hg/Python-2.5.4/Include/phelper.d: line 110: relocation
remains against user symbol D``PyEval_EvalFrameEx (offset 0x5)

I also tried running this without -DPHELPER as a regular DTrace script
rather than a ustack helper and ran into a problem that I've noticed
with any OS X Python build I've tried. I cannot seem to probe any of the
C functions in the Python interpreter. There are no simply
pid$target:a.out:: probes available. I'm wondering if that is an effect
of their being in a .framework, but I think I've been able to probe
other symbols other .frameworks.

Look at these lines at the beginning of phelper.d:

#if defined(__i386)
#define        startframe PyEval_EvalFrameEx
#define        endframe PyEval_EvalCodeEx
#elif defined(__amd64)
#define        PyEval_EvalFrameEx PyEval_EvalFrameExReal
#define        startframe PyEval_EvalFrameExReal
#define        endframe PyEval_EvalCodeEx
#elif defined(__sparc)
#define        PyEval_EvalFrameEx PyEval_EvalFrameExReal
#define        startframe PyEval_EvalFrameEx
#define        endframe PyEval_EvalFrameExReal
#endif

You may need to adjust these if your compiler defines
something different from the __i386 or __amd64.
I guess an

#else
#error your architecture was not recognized
#endif

would be useful here.

This is on an Intel Core 2 Duo running OS X 10.5.6. __i386 is defined.

Robert, I have no idea how Mac OS does pstack helpers without generating
object files, sorry.

no simply pid$target:a.out:: probes available.

Hmm. Try adding -Z to see if that helps.

/Users/rkern/hg/Python-2.5.4/Include/phelper.d: line 110: relocation
remains against user symbol D``PyEval_EvalFrameEx (offset 0x5)

This is trying to tell you that there's no such function. Indeed, this
isn't mentioned in my original patches, have you been editing it?

2010/10/25 Jesús Cea Avión <report@bugs.python.org>:
..

Another question: I am not able to decide between Sun/Apple style, or breaking dtrace scripts
compatibility completely. Anybody has an opinion about this?. Is this actually important?. Are
there so many legacy dtrace scripts out there?.

I would say compatibility with Sun/Apple probes should not stand in
the way of implementing this in cpython. Of course familiarity to
existing users is a consideration, but in cases where self-consistency
can be improved, I don't think we should be overly concerned about
legacy scripts. I would estimate that 90% of future users will never
have used either Sun or Apple probes, 9% will have used Apple and 1%
Sun. (This is completely unscientific guess, of course.)

Updated version of the patch; this ought to contain Include/pydtrace.d:

$ diffstat /tmp/py3k-add-systemtap-2010-10-25.patch
 Include/pydtrace.d         |   10 
 Lib/test/test_systemtap.py |   89 ++++++
 Makefile.pre.in            |   10 
 Python/ceval.c             |   75 +++++
 configure                  |  576 +++++++++++++++++++++++

configure.in | 34 ++
pyconfig.h.in | 3
7 files changed, 522 insertions(+), 275 deletions(-)

Patch contains FIXMEs (sorry), which clearly need addressing.

As for the objectives, do folks here agree with the "Performance" notes in http://bugs.python.org/issue4111#msg100173 ?

Malcolm, does your last patch address the performance issue?. Ideally, dtrace support should be compiled in by default, so performance issues are important.

Idealy, performance difference between compiling dtrace or not should be negligible. Until you actually use it, of course.

We need some documentation, too.

Maybe a new chapter, or a new section in the debug chapter. Better the first, since this is not only for debugging, but for performance study too.

Compiling the code WITHOUT dtrace support gives an error:

"""
Undefined first referenced
symbol in file
__dtrace_python___function__entry ./libpython3.2m.so
__dtraceenabled_python___function__return ./libpython3.2m.so
__dtrace_python___function__return ./libpython3.2m.so
__dtraceenabled_python___function__entry ./libpython3.2m.so
ld: fatal: Symbol referencing errors. No output written to python
collect2: ld returned 1 exit status
"""

Compiling WITH dtrace... shows the same error :-????

I am using Solaris 10, but the configure script detects "apple" implementation.

I believe the problem jcea is experiencing is with the
solaris (/linux?) branch of the configure.in:

if dtrace -G -o /dev/null -s $srcdir/Include/pydtrace.d 2>/dev/null

It seems solaris doesn't like the -o /dev/null part. Try
specifying some real temporary file name instead.

Updated patch, removing the FIXMEs, and slightly reworking the test code.

I've wrapped the whole of get_frame_marker_info with a PyErr_Fetch/PyErr_Restore pair: the PyUnicode_AsUTF8String calls could fail with a MemoryError, and we don't want to confuse the regular exception handling within ceval.

I'm not sure how to write a unit test to test for this: perhaps we could corrupt the __code__ instance so that co_filename is not a PyUnicodeObject, leading to a TypeError within the function, but that's a readonly attribute. Any ideas?

I've also added a unit test for a non-ASCII script (Lib/test/systemtap_sample_☠.py), containing a non-ASCII identifier (文字化け).

The non-ASCII script name (Lib/test/systemtap_sample_☠.py) may be controversial: do we have anything like that in the source tree yet? Is there any risk of messing up the build across platforms, or of impacting the Hg migration?

Still to-do:

• address the "Undefined symbol" issues seen by jcea.
• documentation
• doublecheck performance

I've wrapped the whole of get_frame_marker_info with a
PyErr_Fetch/PyErr_Restore pair: the PyUnicode_AsUTF8String calls could
fail with a MemoryError, and we don't want to confuse the regular
exception handling within ceval.

If PyUnicode_AsUTF8String() is meant to encode a filename, you should
use PyUnicode_EncodeFSDefault() instead.

The non-ASCII script name (Lib/test/systemtap_sample_☠.py) may be
controversial: do we have anything like that in the source tree yet?
Is there any risk of messing up the build across platforms, or of
impacting the Hg migration?

It would be better to generate the sample dynamically, so that users
with an incompatible locale or filesystem aren't prevented from checking
out the source.

It would be better to generate the sample dynamically, so that users
with an incompatible locale or filesystem aren't prevented from checking
out the source.
Thanks: am attaching updated patch: I've removed Lib/test/systemtap_sample_☠.py, and now generate a similarly-named file during the test, using test.support.TEST_FN and unlink

Still TODO:

• address pitrou's concerns about PyUnicode_AsUTF8String from msg119580
• address the "Undefined symbol" issues seen by jcea (msg119563 onwards)
• documentation
• doublecheck performance
• perhaps add a systemtap "tapset", and demo code using it (like I did in Fedora's python3 RPMs)
• anything else I've missed :)

I should note that I've only ever been testing this with SystemTap, on Linux. I don't have a box with DTrace, and I've never directly used it. I wouldn't today be able to diagnose a buildbot failure related to DTrace (I believe I would with systemtap, fwiw).

Are there any DTrace experts around who would be willing to handle the DTrace side of this? If not, would it be reasonable to make this issue be only explicitly about "systemtap"? (e.g. perhaps change the "configure" argument accordingly?) Alternatively, given that this bug originally started as an RFE about DTrace, should we split out systemtap as a separate RFE?

I'm sorry if I've "muddied the waters" by doing this.

For example, the only test coverage I've written (test_systemtap.py) checks for the presence of a "stap" executable, and skips the tests if it's not found. I'm not familiar enough with DTrace to create the same for DTrace.

FWIW (in response to IRC question): "thread_indent" is documented here:
http://sourceware.org/systemtap/SystemTap_Beginners_Guide/systemtapscript-handler.html#thread_indent

It looks like it may be systemtap-specific; however the only usage is within test_systemtap.py, guarded by the presence of a "stap" binary, skipping the tests if it is not found.

For the record, replacing /dev/null with conftest.out in the configure test solves the detection problem (and allows Python to build cleanly). However, there is then a problem in test_systemtap (even when replacing "stap" with "dtrace") since the syntax for scripts doesn't seem compatible.

In any case, the trivial fix for configure:

diff -r 777b171a63ae -r 1784ac25b52e configure
--- a/configure Tue Oct 26 18:50:46 2010 +0200
+++ b/configure Tue Oct 26 18:54:09 2010 +0200
@@ -9178,7 +9178,7 @@
 $as_echo "$with_dtrace" >&6; }
 if test ! -z "$with_dtrace"
 then
-    if dtrace -G -o /dev/null -s $srcdir/Include/pydtrace.d 2>/dev/null
+    if dtrace -G -o conftest.out -s $srcdir/Include/pydtrace.d 2>/dev/null
     then
 
 $as_echo "#define WITH_DTRACE 1" >>confdefs.h
diff -r 777b171a63ae -r 1784ac25b52e configure.in
--- a/configure.in      Tue Oct 26 18:50:46 2010 +0200
+++ b/configure.in      Tue Oct 26 18:54:09 2010 +0200
@@ -2466,7 +2466,7 @@
 AC_MSG_RESULT([$with_dtrace])
 if test ! -z "$with_dtrace"
 then
-    if dtrace -G -o /dev/null -s $srcdir/Include/pydtrace.d 2>/dev/null
+    if dtrace -G -o conftest.out -s $srcdir/Include/pydtrace.d 2>/dev/null
     then
        AC_DEFINE(WITH_DTRACE, 1, 
         [Define if you want to compile in Dtrace support])

(my last message was about building on OpenSolaris)

Dave, we need some kind of documentation, if we expect to ship this in Python 3.2. The deadline is only 10-15 days away.

Could you write something able to be in the standard documentation?.

configure.in has:

AC_MSG_RESULT([$with_dtrace])
...
AC_MSG_RESULT($with_dtrace)

Why twice? It looks confusing.

Some references to keep around:

http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/
http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/Python26-07-dtrace.diff
http://src.opensolaris.org/source/xref/jds/spec-files/trunk/patches/Python-07-dtrace.diff

Some more references:

Read the notes under the slides:
https://dgl.cx/2011/01/dtrace-and-perl

https://dgl.cx/dtrace

http://dtrace.org/blogs/

What do we need to unblock this?

2011/4/6 Jesús Cea Avión <report@bugs.python.org>:

Jesús Cea Avión <jcea@jcea.es> added the comment:

Some more references:

Read the notes under the slides:
https://dgl.cx/2011/01/dtrace-and-perl

https://dgl.cx/dtrace

http://dtrace.org/blogs/

What do we need to unblock this?

Summarize 30+ page discussion in a new issue.
Blog about it on http://blog.python.org/

anatoly t.

jcea: I notice that on 2011-02-22 you made these changes:
assignee: dmalcolm -> dino.viehland
nosy: +dino.viehland
versions: +Python 3.3 -Python 3.2

Did you mean to change the assignee, or was this an accident?

Malcolm, it was a mistake. I only wanted to change the TARGET.

I assign the issue to you again.

Dino, I delete you from the nosy list now. Sorry for the inconvenience.

My excuses to both.

Anybody still working on this?.

We missed the 2.7 boat. DO NOT MISS THE 3.3 ONE!!! :-)

This project continues in issue bpo-13405.

Issue bpo-13405 covers DTrace; I've taken the liberty of also opening issue bpo-14776 to cover SystemTap. I hope that once one of these is in the tree it will be easier to get the other one in.