trying to do a

hotshot.stats.load("myprofiling_file.prof")

fails with assertionerror

assert not self._stack

python 2.3.2 on WinXP

Logged In: YES
user_id=469548

While the original report isn't very useful, I've ran into
this problem multiple times as well. I can reproduce it
using the attached profile file (compressed because of the
large size) and the following console session:

Python 2.3.4 (#2, Jul  5 2004, 09:15:05)
[GCC 3.3.4 (Debian 1:3.3.4-2)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> import hotshot.stats
>>> stats = hotshot.stats.load('roundup.prof')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.3/hotshot/stats.py", line 12, in load
    return StatsLoader(filename).load()
  File "/usr/lib/python2.3/hotshot/stats.py", line 51, in load
    assert not self._stack
AssertionError
>>>

I'm not sure who's baby hotshot really is, so I'm leaving
this unassigned.

Logged In: YES
user_id=469548

Hmm, the file was too big, even though it was compressed.
I've uploaded it to
http://home.student.uva.nl/johannes.gijsbers/roundup.prof.bz2
now.

Logged In: YES
user_id=12800

Could this be related to 1019882?

Logged In: YES
user_id=86307

I ran into this today, so I decided to look into it. It
looks to me like the problem only happens if you profile
with lineevents enabled. In that case, hotshot uses the
tracer_callback function (in _hotshot.c) to dispatch trace
events. This function explicitly ignores exception returns
(PyTrace_EXCEPTION), which can lead to an unbalanced stack
of calls/returns when the log is loaded (if an profiled
function exits with an exception).

It seems on the surface that tracer_callback ought to handle
exceptions the same way as normal returns. This would be
consistent with what happens when profiler_callback is used,
since PyEval_EvalFrame dispatches sends a Py_RETURN to
c_profilefunc when exiting because of an exception.

Logged In: YES
user_id=86307

Well, the superficial fix doesn't work (sorry for posting
too soon). It turns out that PyTrace_EXCEPTION is sent for
any exception, not just one causing the function to exit.
So I guess the best fix may be to have hotshot always
install its profiler_callback to handle CALLS and RETURNS,
and then optionally install the tracer_callback to handle
only LINEs. Anyway, that works for the one test case I've
been using (a runcall of a function which simply does
"import pickle").

By the way, I'm testing with 2.4b1.

Logged In: YES
user_id=12800

See 900092-patch.txt for a candidate patch against Python
2.4.1. Props to Justin Campbell for most of the heavily
lifting (but you can blame me for any problems ;).

This fix restore the tracing of a 'return' event for
exceptions that cause a function to exit.

Logged In: YES
user_id=12800

900092-patch-2.txt fixes the test suite for the extra return
event.

Logged In: YES
user_id=12800

Patches applied for Python 2.4.2 and 2.5a1

Has this ever been reported again since it was closed ? I just got it today (python 2.5). The prof file is 11MB, I'll see if I can reproduce the bug with a smaller one.

Ok, I reduced the prof file to 38K. Can I upload an attachment here ? I don't see how.

I think only OPs and project members can upload. Email it to me, George (brett at python.org), mention the bug # and I will upload it.

Attaching the file George made.
File Added: bug.prof

Has test, patch and sample data for reproducing.

I added some prints in the hotshot.stats.load function, this lead to
interesting things:

• first, the program seems to spawn subprocesses which trace themselves
  in the same file.

• however, there is real issue when fork() is called on a traced
  program: calls to _execvpe should not appear in the trace!
  IMO PyOS_AfterFork should disable any trace or profile function.

Is it worth leaving this open, given this from section 26.5 of the 2.7 docs. "For common usage it is recommended to use cProfile instead. hotshot is not maintained and might be removed from the standard library in the future."

Closing as hotshot is not maintained and is not documented in py3k.

I don't think this should have been closed just yet. If the issue still exists in 2.x, it could still be fixed in the remaining 2.6 release, or any of the future 2.7 releases.

You are right that it won't apply to 3.x since hotshot is gone there.

Who of our overworked volunteers is going to do the work, given that hotshot doesn't even feature on the maintainers list?

Closing as no reply to msg110827.

It was brought to my attention that this is still an issue and that it was closed. Reopening.

IMHO reopening this was a complete and utter waste of core devs time, its already passed its tenth birthday!!!

It remains a valid issue. Time is not a factor.

You clearly haven't bothered to read that there is *NO* maintainer, that the docs for 2.x recommend that you *DON'T* use it, that its been removed from 3.x and that nobody has yet volunteered to do anything with it. If its so important, why don't you do the work instead of trying to palm it off onto people who have far more important things to consider? I'll therefore repeat so that you very clearly get my message, this is a complete and utter waste of core devs time.

Brian Curtin is a core developer, perhaps you should trust his judgement.

Then God help Python is all I can say.