classification
Title: perf_counter result does not count system sleep time in Mac OS
Type: behavior Stage: patch review
Components: macOS Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, jab, ned.deily, njs, nooB, p-ganssle, ronaldoussoren, vstinner, xrisk
Priority: normal Keywords: patch

Created on 2020-07-15 12:43 by nooB, last changed 2020-08-05 14:59 by vstinner.

Pull Requests
URL Status Linked Edit
PR 21719 closed xrisk, 2020-08-03 22:01
Messages (8)
msg373690 - (view) Author: nooB (nooB) Date: 2020-07-15 12:43
Documentation for time.perf_counter says "does include time elapsed during sleep".

https://docs.python.org/3.8/library/time.html#time.perf_counter

But it does not work as expected in Mac OS. I learnt that perf_counter uses the underlying c function: "mach_absolute_time".

-------------------------

import time
time.get_clock_info('perf_counter')
namespace(adjustable=False, implementation='mach_absolute_time()', monotonic=True, resolution=1e-09)

-------------------------

The documentation for "mach_absolute_time" clearly states that "this clock does not increment while the system is asleep"

https://developer.apple.com/documentation/kernel/1462446-mach_absolute_time


FWIW, Mac kernel does offer another function which returns monotonic ticks "including while the system is asleep"

https://developer.apple.com/documentation/kernel/1646199-mach_continuous_time

But it seems to be available only on Mac 10.12+.
msg374523 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-07-28 19:37
macOS 10.12 also has clock_gettime(), which would allow using the generic code path (although the current path must be kept for older versions of macOS).
msg374765 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-03 22:55
perf_counter documentation says "It does include time elapsed during sleep and is system-wide." where "sleep" here means time.sleep():
https://docs.python.org/dev/library/time.html#time.perf_counter

Python clock functions don't provide any warranty regarding to system suspend or system hibernation. See PEP 418 for more details: "The behaviour of clocks after a system suspend is not defined in the documentation of new functions. The behaviour depends on the operating system: see the Monotonic Clocks section below."
https://www.python.org/dev/peps/pep-0418/#monotonic-clocks

I don't think that using mach_continuous_time() is needed, but the documentation should be clarified. The doc should explain that the behavior during system suspend is not defined (platform specific).
msg374840 - (view) Author: Rishav Kundu (xrisk) * Date: 2020-08-04 18:56
Wouldn’t using mach_continuous_time (and its equivalents on other platforms) wherever possible be preferable?

Or would a different API that distinguishes between clocks that track during suspend versus those that not be a better idea? Given that at least macOS and Linux offer both variants (not sure about Windows)
msg374852 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-05 00:29
> Or would a different API that distinguishes between clocks that track during suspend versus those that not be a better idea? Given that at least macOS and Linux offer both variants (not sure about Windows)

I don't think that all platforms provide a clock for Python time.perf_counter() which include suspend. So I'm not sure that it's useful to only change the behavior on macOS, and only depending on the macOS version (and macOS target version).

Maybe a new clock is needed, clock which has a well defined behavior for system suspend, on any platform. Such clock may not be available on all platforms.

FYI on Python 3.3, time.monotonic() was not available on all platforms. It became available on all platforms on Python 3.5.
https://docs.python.org/dev/library/time.html#time.monotonic
msg374853 - (view) Author: Rishav Kundu (xrisk) * Date: 2020-08-05 00:42
> Maybe a new clock is needed, clock which has a well defined behavior for system suspend, on any platform.

I’d like to work on this, if possible. Linux and macOS support seems to be straightforward. I will have to look into other platforms.

What would be the protocol exactly? Should I float a discussion on the ideas mailing list? Open a new bpo?
msg374854 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2020-08-05 01:08
I made a record of my investigations here: https://github.com/python-trio/trio/issues/1586

One reason we might care about this is that asyncio ought to be using a clock that stops ticking while suspended.

(On which note: Please don't switch macOS to use clock_gettime(CLOCK_MONOTONIC); that would be a breaking change for us!)

At least Linux, macOS, FreeBSD, and Windows all have a way to access a monotonic clock that stops ticking while the system is suspended.
msg374888 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-05 14:59
> (On which note: Please don't switch macOS to use clock_gettime(CLOCK_MONOTONIC); that would be a breaking change for us!)

Right, I propose to add new clock(s) since people may rely on the current clock(s) specifications.

> At least Linux, macOS, FreeBSD, and Windows all have a way to access a monotonic clock that stops ticking while the system is suspended.

Python also tries to support AIX, OpenBSD, Solaris, Android, etc.
https://pythondev.readthedocs.io/platforms.html#best-effort-and-unofficial-platforms

I'm not sure that all "supported" platforms provide such clock. It's ok if such clock is not available on all platforms. People can fallback on monotonic/perf_counter depending on their need. For example, previously, it was common to write something like:

try: from time import monotonic
except ImportError: from time import time as monotonic # Python 2 or clock not available
History
Date User Action Args
2020-08-05 14:59:41vstinnersetmessages: + msg374888
2020-08-05 01:08:07njssetmessages: + msg374854
2020-08-05 00:46:26jabsetnosy: + jab
2020-08-05 00:42:30xrisksetmessages: + msg374853
2020-08-05 00:29:37vstinnersetnosy: + njs
messages: + msg374852
2020-08-04 18:56:25xrisksetmessages: + msg374840
2020-08-03 22:55:51vstinnersetmessages: + msg374765
2020-08-03 22:01:54xrisksetkeywords: + patch
nosy: + xrisk

pull_requests: + pull_request20862
stage: patch review
2020-07-28 19:37:06ronaldoussorensetmessages: + msg374523
2020-07-15 12:48:51ned.deilysetnosy: + belopolsky, vstinner, p-ganssle
2020-07-15 12:43:19nooBcreate