New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implementation of the PEP 418 #58636
Comments
Attached patch implements the PEP-418: add time.monotonic() and time.hires(). In practice, it replaces time.steady() by time.hires() and time.steady(strict=True) by time.monotonic(). |
Tiny review. Two more details:
|
Updated patch:
TODO: time.monotonic() must use GetTickCount64() or GetTickCount() on Windows, with a detection of integer overflow on GetTickCount(). |
Patch version 3:
Clock information on different operating systems. Linux (Fedora 16) on x86_64: >>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 1e-06,
'function': 'clock()',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 1e-06}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 1e-09,
'function': 'clock_gettime(CLOCK_MONOTONIC_RAW)',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 1e-09,
'function': 'clock_gettime(CLOCK_MONOTONIC_RAW)',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 1e-09,
'function': 'clock_gettime(CLOCK_REALTIME)',
'is_adjusted': True,
'is_monotonic': False,
'resolution': 1e-09} FreeBSD 8.2 on x86_64: >>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 0.0078125,
'function': 'clock()',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 0.0078125}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 1.1000000000000001e-08,
'function': 'clock_gettime(CLOCK_MONOTONIC)',
'is_monotonic': True,
'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 1.1000000000000001e-08,
'function': 'clock_gettime(CLOCK_MONOTONIC)',
'is_monotonic': True,
'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 1.1000000000000001e-08,
'function': 'clock_gettime(CLOCK_REALTIME)',
'is_adjusted': True,
'is_monotonic': False,
'resolution': 1e-09} OpenBSD on x86_64: >>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 0.01,
'function': 'clock()',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 0.01}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 0.01,
'function': 'clock_gettime(CLOCK_MONOTONIC)',
'is_monotonic': True,
'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 0.01,
'function': 'clock_gettime(CLOCK_MONOTONIC)',
'is_monotonic': True,
'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 0.01,
'function': 'clock_gettime(CLOCK_REALTIME)',
'is_adjusted': True,
'is_monotonic': False,
'resolution': 1e-09} Python 32 bits on Windows 64 bits: >>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 1e-08,
'function': 'QueryPerformanceCounter()',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 1e-08}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 1e-08,
'function': 'QueryPerformanceCounter()',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 1e-08}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 0.015600099999999999,
'function': 'GetTickCount()',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 0.001}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 0.015600099999999999,
'function': 'GetSystemTimeAsFileTime()',
'is_adjusted': True,
'is_monotonic': False,
'resolution': 1e-07} Some remarks:
|
Solaris on x86_64: >>> pprint.pprint(time.get_clock_info('clock'))
{'accuracy': 1e-06,
'function': 'clock()',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 1e-06}
>>> pprint.pprint(time.get_clock_info('highres'))
{'accuracy': 2e-09,
'function': 'clock_gettime(CLOCK_HIGHRES)',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('monotonic'))
{'accuracy': 2e-09,
'function': 'clock_gettime(CLOCK_HIGHRES)',
'is_adjusted': False,
'is_monotonic': True,
'resolution': 1e-09}
>>> pprint.pprint(time.get_clock_info('time'))
{'accuracy': 0.01,
'function': 'clock_gettime(CLOCK_REALTIME)',
'is_adjusted': True,
'is_monotonic': False,
'resolution': 1e-09} |
TODO (PEP-418-3.patch):
|
Hi Victor, I think you need to reconsider the time.steady() name you're using The fallback to time.time() is not a good idea, since then the programmer Regardless of this functional problem, I'm also not sure what you want Instead of trying to tweak all the different clocks and timers into BTW: Thanks for the research you've done on the different clocks and Thanks,Marc-Andre Lemburg 2012-04-03: Python Meeting Duesseldorf today ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 |
I opened a new thread on python-dev to discuss this topic.
Oh, I should explain this choice in the PEP. Basically, the idea is to
Nope, time.get_clock_info('steady') does not change at runtime. So it
This is a completly different approach. It should be discussed on
You're welcome. |
STINNER Victor wrote:
With "every time" I meant: in every application you use the function. It's better to use a try-except to test for availability of
Fair enough. BTW: Are aware of the existing systimes.py module in pybench, |
Patch version 4:
|
Nope, I didn't know it. It mentioned it in the PEP. |
Patch version 5, updated to the last version of the PEP:
|
+#if defined(linux) || defined(__linux) || defined(linux) Hum, a better check should be done in configure and/or a macro like MS_WINDOWS should be added. |
Version 6: simplify the code handling GetTickCounter() integer overflow. We don't need a has_last_ticks variable. |
+ if (has_getickcount64) { ";" is missing after "1e-3", it does not compile on Windows because of this. |
Patch version 7:
|
perf_counter_process_time.patch: replace "time.clock if windows else time.time" with time.perf_counter, and getrusage/clock with time.process_time. pybench and timeit now use time.perf_counter() by default. profile uses time.proces_time() by default. pybench uses time.get_clock_info() to display the precision and the underlying C function (or the resolution if the precision is not available). Tools/pybench/systimes.py and Tools/pybench/clockres.py may be removed: these features are now available directly in the time module. |
STINNER Victor wrote:
No changes to the pybench defaults, please. It has to stay backwards Thanks,Marc-Andre Lemburg 2012-04-28: PythonCamp 2012, Cologne, Germany 15 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 |
Patch version 8: time.process_time() uses times() if available. Rename also "function" key of time.get_clock_info() with "implementation". |
Patch version 9: fixes for Windows (fix compilation and fix to code checking if GetTickCount64 is present). |
The precision of mach_absolute_time() is known: it is timebase.numer / timebase.denom * 1e-9. |
FreeBSD doesn't provide CLOCK_PROCESS_CPUTIME_ID, but CLOCK_PROF. CLOCK_PROF can be used instead of getrusage(), its precision can be read using clock_getres(). Something like "> #if defined(CLOCK_PROF) && defined(FreeBSD)" can be used. (By the way, "#if defined(linux)" may be enough) I read somewhere than CLOCK_PROF is the user+system CPU time of the *current thread* on OpenBSD. I can be checked by the following unit test: def test_process_time_threads(self):
class BusyThread(threading.Thread):
def run(self):
timeout = monotonic() + 1.0
loops = 1
while monotonic() < timeout:
i = 0
while i < loops:
i += 1
loops *= 2
t1 = process_time()
thread = BusyThread()
thread.start()
thread.join()
t2 = process_time()
self.assertGreater(t2 - t1, 0.9) -- perf_counter() should remember if win32_perf_counter() failed or not, as the pseudo-code of the PEP. I prefer to leave clock() unchanged to keep backward compatibility. |
Victor, can you let us know when you think the patch is ready for |
Now! I created a repository for the PEP. I integrated my own last comments. I tested the PEP on Linux 3.3, FreeBSD 8, OpenBSD 5, OpenSolaris and Windows Seven. The implementation is ready for a review. The last patch is 384190bb0bd5.diff (presented as "Patch Set 12" in Rietveld). |
Please leave the pybench default timers unchanged in case the The perf_counter_process_time.patch currently changes them, even Ditto for the resolution changes: these need to be optional and Thanks,Marc-Andre Lemburg 2012-04-28: PythonCamp 2012, Cologne, Germany 10 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 |
Ok, done in the new patch: perf_counter_process_time-2.patch. |
STINNER Victor wrote:
Thanks. |
667541bb315c.diff: Updated patch, last change: is_adjusted key of time.get_clock_info() is now mandatory. |
New changeset 76d2e0761d18 by Victor Stinner in branch 'default': |
Guido van Rossum accepted the PEP, let's commit the implementation. |
New changeset bd195749c0a2 by Victor Stinner in branch 'default': |
test_process_time_threads is failing on one of the buildbots: Traceback (most recent call last):
File "/var/lib/buildbot/buildarea/3.x.warsaw-ubuntu-arm/build/Lib/test/test_time.py", line 408, in test_process_time_threads
self.assertGreater(t2 - t1, 0.1)
AssertionError: 0.08041412500006118 not greater than 0.1
""" The test is a bit too optimistic: a thread is looping for 0.2s, and the test checks that the process time (user + system) increased of at least 0.1s. If the thread is preempted, there's a high chance you won't get even as low as 0.1s. |
New changeset 1255cac63dfc by Victor Stinner in branch 'default': |
"The test is a bit too optimistic: a thread is looping for 0.2s, and the test checks that the process time (user + system) increased of at least 0.1s. If the thread is preempted, there's a high chance you won't get even as low as 0.1s." Hum, the problem is maybe that the thread is preempted, but the first problem is that the test considers that time.process_time() uses seconds. I rewrote the test to make it more reliable. |
Hum, what? |
There are many sporadic failures on the buildbots: ====================================================================== Traceback (most recent call last):
File "/usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/test_time.py", line 441, in test_process_time_threads
self.assertGreaterEqual(t2 - t1, busy)
AssertionError: 0.01249799999999368 not greater than or equal to 0.013093999999966854 |
New changeset 0bdf0727ee29 by Victor Stinner in branch 'default': |
New changeset ad3d6010379b by Victor Stinner in branch 'default': |
Yes, dt is not a number of seconds in the following example: t1=time.process_time(); (...); t2=time.process_time(); dt=t2-t1
I wrote the test to check if time.process_time() measures the total CPU of the process, and not the CPU time of only the current thread. I'm tired of this PEP, so I just removed the test. I don't think that it is really interesting and it looks difficult to write a reliable test. |
OK, so what is it then? Furthermore: rt = time.time()
pt = time.process_time()
for i in range(int(sys.argv[1])):
pass print("real time: %f" % (time.time() - rt)) It really looks like seconds to me, definitely not jiffies ;-)
Hum, right now, the only process_time test I see is test_process_time, """ It's not that process_time() is not precise (I assume you mean accurate ;-). |
time.process_time() uses maybe "seconds" on Linux, but it doesn't include time elapsed during sleep. See the test: def test_process_time(self):
start = time.process_time()
time.sleep(0.1)
stop = time.process_time()
self.assertLess(stop - start, 0.01) According to Wikipedia: "Since 1967, the second has been defined to be: the duration of 9,192,631,770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the caesium 133 atom." The caesium 133 atom is not sleeping while your process is sleeping, so you cannot say the time.process_time() uses second. Do you see what I mean? |
I'm closing again this issue. @neologix: Please open a new issue if you disagree with me on the definition of "seconds" for time.process_time(). |
I won't reopen, but I still disagree with your definition. """ real 0m1.014s AFAICT, process_time() returns user + sys. Since the unit of those two |
I don't know what you two are arguing about, since the process_time() doc says "Return the value (in fractional seconds) of the sum of the system and user CPU time of the current process". (while I'm not sure what "fractional" seconds are, they are probably seconds in the first place :-)) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: