classification
Title: pystone.py in 3.4 still uses time.clock(), even though it's marked as deprecated since 3.3
Type: enhancement Stage:
Components: Benchmarks Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: fdrake, pfalcon, pitrou, python-dev, serhiy.storchaka, vstinner, yselivanov
Priority: normal Keywords:

Created on 2014-02-01 21:10 by pfalcon, last changed 2014-06-04 20:50 by vstinner. This issue is now closed.

Messages (13)
msg209916 - (view) Author: Paul Sokolovsky (pfalcon) * Date: 2014-02-01 21:10
http://docs.python.org/3.3/library/time.html#time.clock says that it's deprecated, but pystone.py in Python-3.4.0b3 tarball still uses it. 

Please kindly consider switching it to plain time.time() and not to some other novelties.

My usecase is: I'm working on alternative Python implementation, I of course want to benchmark it, and of course want to claim that I ran unmodified pystone.py. Now to achieve that, I need to implement deprecated function.
msg209944 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-02-02 02:42
Well, I guess we can replace

    from time import clock

with something like

    try:
        from time import monotonic as clock
    except ImportError:
        from time import time as clock

Victor, what do you think?
msg209959 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-02-02 08:07
time.monotonic() has a bad resolution (15.6 ms) on Windows. To measure
performances, use time.perf_counter() or time.process_time() as proposed in
tome.clock() documentation. It depends if you want to include time elapsed
during sleep. Usually time.perf_counter() is the expected function.
msg210035 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-02-02 22:16
The problem with pystone is that such tool is used to compare performances between different versions of Python. If pystone uses a different clock in Python 3.4, you may not be able to compare compare results with older Python versions.

In my opinion, such change should not be done before Python 3.5. 3.4 beta 3 has already been released.
msg210039 - (view) Author: Paul Sokolovsky (pfalcon) * Date: 2014-02-02 22:28
> The problem with pystone is that such tool is used to compare performances between different versions of Python.

That's why I just propose to switch it to time.time(), which surely is available on each and every Python version and implementation. If there's a concern that some implementation may have only 1-second precision, then: 1) well, pystone has limits to its "scientificity", it's more like quick-run-anywhere, there's pybench for "real" testing (it's maintained and supported I hope); 2) number of iteratations can be bumped from 50K to 200K-500K.
msg210040 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-02-02 22:31
> That's why I just propose to switch it to time.time(), which surely is available on each and every Python version and implementation.

It's not the same clock. time.clock() measures the process time, which is different from the wall clock time.

Compare time.get_clock_info('clock') to time.get_clock_info('time') and time.get_clock_info('perf_counter').

See also the PEP 418 which lists all these clocks and explain why time.clock() was deprecated:
http://www.python.org/dev/peps/pep-0418/
msg210046 - (view) Author: Paul Sokolovsky (pfalcon) * Date: 2014-02-02 22:42
Yes, and my note about "scientificity" above. Also compare with your own account of time.perf_counter() above ("Usually time.perf_counter() is the expected function.")

Also, I didn't want to start discussion on how to do benchmarking right, just to point out a small inconsistency, and suggest a fix which balances on (down-to-earth) correctness and practicality. It even can be ignored, except that such small inconsistencies accumulate and then some time later stick from every hole if not addressed.
msg210055 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-03 00:12
> If pystone uses a different clock in Python 3.4, you may not be able 
> to compare compare results with older Python versions.

This sounds a bit silly. Typical pystone runs last a couple seconds at most, there won't be a significant clock drift in such a short amount of time.

Besides, if time.clock() is deprecated, pystone will *have* to be modified one day, anyway.
msg210056 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-03 00:14
Paul:

> well, pystone has limits to its "scientificity", it's more like quick-
> run-anywhere, there's pybench for "real" testing (it's maintained and 
> supported I hope)

pybench is so synthetic it's almost a nano-benchmark. I would suggest using the benchmarks suite, which will give more useful numbers IMHO:
http://hg.python.org/benchmarks/
msg210057 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-02-03 00:17
> Besides, if time.clock() is deprecated, pystone will *have* to be modified one day, anyway.

If something is changed, I would prefer to simply drop this old tool. It is not reliable and so may give you wrong results.
msg210083 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-02-03 07:13
Pystone should has an option to specify a timer (as timeit).
msg219768 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-06-04 18:45
New changeset 38acef3c3228 by Guido van Rossum in branch 'default':
Replace deprecated time.clock() with time.time(). Fixes issue #20475.
http://hg.python.org/cpython/rev/38acef3c3228
msg219771 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-04 20:50
Instead of time.time, time.perf_counter would be better.
History
Date User Action Args
2014-06-04 20:50:04vstinnersetmessages: + msg219771
2014-06-04 18:45:48gvanrossumsetstatus: open -> closed
resolution: fixed
2014-06-04 18:45:07python-devsetnosy: + python-dev
messages: + msg219768
2014-02-03 07:13:47serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg210083
2014-02-03 00:17:04vstinnersetmessages: + msg210057
2014-02-03 00:14:18pitrousetmessages: + msg210056
2014-02-03 00:12:58pitrousetnosy: + pitrou
messages: + msg210055
2014-02-02 22:42:49pfalconsetmessages: + msg210046
2014-02-02 22:31:31vstinnersetmessages: + msg210040
2014-02-02 22:28:06pfalconsetmessages: + msg210039
2014-02-02 22:16:47vstinnersetmessages: + msg210035
versions: + Python 3.5, - Python 3.3, Python 3.4
2014-02-02 08:07:11vstinnersetmessages: + msg209959
2014-02-02 02:55:42yselivanovsetnosy: + fdrake
2014-02-02 02:42:57yselivanovsetnosy: + yselivanov
messages: + msg209944
2014-02-01 21:53:39vstinnersetnosy: + vstinner
2014-02-01 21:10:36pfalconsetcomponents: + Benchmarks
versions: + Python 3.3, Python 3.4
2014-02-01 21:10:18pfalconcreate