cProfile should go away in 3.0 (as a public module) and just be what 
profile uses when available.

I will try to fix issue. I cannot promise that I will get it done before
the beta though.

This didn't get done for the betas, but we're still going to release. 
Knocking it down to critical.

Can you get this done before beta 3?

Sorry, I can't. I would love to get this done, but I am just too busy
this week (and next week doesn't look any better). Furthermore, the
amount of work that the profile/cProfile merge will require is quite
considerable. So, I don't expect someone else could get this done before
the beta.

Let's retarget 3.1 then.

How is this going?

Attached is *an* attempt to combine the old profile/cProfile modules into a single 
one called profile.  Only the C and Python source and test files are included, not 
any documentation files.  More details are in the README file, copied below.

Hopefully, this is useful.

/Jean Brouwers


These are the modified files to replace the old profile module with the fast cProfile 
one and rename the latter.  A complete list and description follow.


./README

./profileNEW/setup.py
./profileNEW/Lib/profile.py
./profileNEW/Lib/test/profilee.py
./profileNEW/Lib/test/test_profile.py
./profileNEW/Modules/_profile.c

./profilePy3k/setup.py
./profilePy3k/Lib/profile.py
./profilePy3k/Lib/test/profilee.py
./profilePy3k/Lib/test/test_cprofile.py
./profilePy3k/Lib/test/test_profile.py
./profilePy3k/Modules/_lsprof.c

./diff_-u_profilePy3k_Modules_lsprof.c_profileNEW_Modules_profile.c.txt
./diff_-ur_profilePy3k_profileNEW.txt

./_profile_hires.c



Directory profileNEW contains all the source and test files in their corresponding 
directories.  Directory  profilePy3k contains the old, original files from the py3k 
trunk taken on May 31, 2009.

Basically, the old _lsprof.c file has been modified and renamed to _profile.c.  Both 
old test_profile.py and test_cprofile.py files have been combined into a single new 
file test_profile.py.

The file  diff_-ur_profilePy3k_profileNEW.txt  contains the forward diffences between 
all the new and orginal files.

File diff_-u_profilePy3k_Modules_lsprof.c_profileNEW_Modules_profile.c.txt contains 
just the differences between the old  _lsprof.c and new  _profile.c files.

Lastly, for your consideration, file _profile_hires.c is the new _profile.c file 
enhanced to use a high resolution timer, see <http://bugs.python.org/issue2281>.  
This file would replace the _profile.c file in the profileNEW/Modules directory.



The new files and tests were installed in a Python 3.0.1 build and compiled and 
tested on MacOS X 10.4.11 Tiger (Intel).  The patches from issue 5330 were also 
installed in this build, see <http://bugs.python.org/issue5330>.  All tests passed 
after regenerating the expected results

% python.exe Lib/test/test_profile.py -r
Regenerating Lib/test/test_profile.py...

% python.exe Lib/test/test_profile.py 
test_bad_counter_during_dealloc (__main__.ProfileTest) ... ok
test_calling_conventions (__main__.ProfileTest) ... ok
test_profile (__main__.ProfileTest) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.017s

OK

Thanks for the code, Jean. With Python 3.1 about to go out the door this
will have to be a 3.2 thing. But I plan to start looking at this module
merge some time in July so I should get to looking at what you did then
(unless someone beats me to it).

I just added another comment about the high-resolution timer in <http://bugs.python.org/issue2281>.

Fixed a couple of typos in the README and _profile_hires.c files of the 
attached profile_module2.tgz tar ball.

Jean Brouwers wrote:
> All tests passed after regenerating the expected results

The tests for profile are output test. If you regenerate them, they pass
for sure.

Merging cProfile/profile will require a lot more work than just renaming
the module. The designs of cProfile and profile are significantly
different. So a successful merge will require settling one profiler
design (cProfile is the better designed one, IMHO) and rewrite the other
one to match the chosen design.

Yesterday, I ported the _lsprof module to Python as an experiment. I
still have a few issues to iron out, but the core of the functionality
is there.  

I am not sure yet how this experiment will fit in the profile/cProfile
merge. I wrote the port mainly to get a better idea of what is needed
for the merge and how the profiler works.

Here's an updated version of my lsprof.py module. I fixed a few bugs,
includes the awful reference leak that was present in the previous
version (i.e., the profiler code was keeping a reference to every frame
executed).

I consider the lsprof.py code functionally equivalent to _lsprof. Now, I
am wondering if I should add the features from profile.py missing in
_lsprof, e.g., the bias argument and the calibration code.

Any comment about the code I posted?

I'll see if I can have a look sometime soon, but my profile experience is practically non-existent so I am not the best person for a code review.

Is there anyone who could review this, it seems a shame that someone's work has been sitting in limbo for over a year?

Is this a case of pure python implementation with a C speedup?  Can you upload the patch to Rietveld?  I'll take a look once it is there.

Alexandre has commit privs so he could just go ahead and commit if he wants.

I do hope to look at this code at some point in my life. I will see if I can't find time at EuroPython next week, especially if Alexander has fixes to it and still feels the need for a review.

So I didn't find time at EuroPython. =) Is this dead, Alexandre, or do you want to see this happen for Python 3.4?

Last desperate ping for 3.4?

Let's close this. There is no point keeping this languishing any further.

The profile and cProfile modules are different enough that we can't merge them without breaking compatibility. They should treated as two separate profilers.

Maybe in the future we should include a drop-in module written in Python for cProfile to be used for another Python implementation. But for now, I don't see the benefit, since the profile module is good enough and alternate implementations should be able to re-do the _lsprof accelerator in their target language without too much trouble if desired.