This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author kristjan.jonsson
Recipients kristjan.jonsson
Date 2010-08-15.15:09:30
SpamBayes Score 2.2021e-11
Marked as misclassified No
Message-id <>
One of the problems with the profiling modules provided with Python is that they are not useful in the presence of multiple threads.  This is because time spent in a different thread may be falsely attributed to some random place in a thread being profiled.

This patch helps fix that, by making the _lsprof.c module (the engine behind cProfile) multi-stack aware.  At every entry into the profiler, a check is made to see which stack is in operation (by looking at the thread state).  The previous stack is then paused and profiling commences on the new stack.

Time spent on other stacks is then subtracted from the measured time on each stack.

A complication arises because it is no longer possible to determine the recursion level of each function (or subcall instance) on each stack by looking at the function's entry alone.  For this reason, it becomes necessary to walk the stack in cases where there are multiple stacks and multiple total recursions seen for the entries.

This patch has been successfully used, with a modifiaction for stackless python, in production at CCP (the modification uses the Tasklet ID rather than the TLS pointer as a key to the stack map).

To be useful, it is important that all threads in the process are set to use the same cProfile.Profiler() instance.  Currently there is no easy way to do that and this patch doesn't attempt to fix that.  But is is possible that an application designed for profiling would attach the profiler at each thread start point.  (In the version of Stackless Python that this is used on, it is possible to enable tracing/profiling of all tasklets simultaneously)
Date User Action Args
2010-08-15 15:09:37kristjan.jonssonsetrecipients: + kristjan.jonsson
2010-08-15 15:09:36kristjan.jonssonsetmessageid: <>
2010-08-15 15:09:34kristjan.jonssonlinkissue9609 messages
2010-08-15 15:09:33kristjan.jonssoncreate